2024-09-21T22:36:43.266 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T22:36:43.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T22:36:43.374 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914492' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOIc4XLKRKA3fh/vlr5hG/eRj+ijFVxgTmfsbyDqzhLo4H9nzQXuFpKibdCq6U05DAN+1rjzG6xkxQtzBpGH4fE= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGT/EY8LA7640P1YsNZ+wmrCB717w/+KCum9Gj/SGN5X3Pfb407TDRJ7NvcSxKK4MKyWqNHcLvu3OI/tI8vB1KQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T22:36:43.374 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-21T22:36:43.375 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-21T22:36:43.375 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T22:36:43.376 INFO:teuthology.task.internal:Checking packages... 2024-09-21T22:36:43.396 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-21T22:36:43.396 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T22:36:43.396 INFO:teuthology.packaging:ref: None 2024-09-21T22:36:43.396 INFO:teuthology.packaging:tag: None 2024-09-21T22:36:43.396 INFO:teuthology.packaging:branch: squid 2024-09-21T22:36:43.396 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:36:43.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-21T22:36:43.596 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:36:43.599 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T22:36:43.606 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T22:36:43.606 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T22:36:43.611 INFO:teuthology.task.internal:Saving configuration 2024-09-21T22:36:43.624 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T22:36:43.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T22:36:43.655 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:31:06.853827', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILlOn/jyC1qH+O860SJ82yQ7TWDtOSbmIo/+hkfrjA5k'} 2024-09-21T22:36:43.675 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:31:06.855476', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFXYSmKfadQb1pByB+FOqT9S1idFatuiS9+o/GDXh4HL'} 2024-09-21T22:36:43.675 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T22:36:43.681 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-21T22:36:43.681 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-21T22:36:43.681 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T22:36:43.742 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f524132b880>, signals=[15]) 2024-09-21T22:36:43.742 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T22:36:43.748 INFO:teuthology.task.internal:Opening connections... 2024-09-21T22:36:43.748 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:36:43.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:36:43.830 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:36:43.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:36:43.904 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T22:36:43.911 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-21T22:36:43.915 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-21T22:36:43.916 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Ubuntu" 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="22.04" 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_CODENAME=jammy 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:ID=ubuntu 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE=debian 2024-09-21T22:36:43.963 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T22:36:43.964 INFO:teuthology.orchestra.run.smithi094.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T22:36:43.964 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T22:36:43.964 INFO:teuthology.orchestra.run.smithi094.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T22:36:43.964 INFO:teuthology.orchestra.run.smithi094.stdout:UBUNTU_CODENAME=jammy 2024-09-21T22:36:43.964 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-21T22:36:44.164 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-21T22:36:44.169 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-21T22:36:44.170 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu" 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="22.04" 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=jammy 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T22:36:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T22:36:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T22:36:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T22:36:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=jammy 2024-09-21T22:36:44.216 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-21T22:36:44.243 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T22:36:44.251 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T22:36:44.256 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T22:36:44.256 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-21T22:36:44.259 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-21T22:36:44.263 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T22:36:44.269 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T22:36:44.269 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-21T22:36:44.304 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-21T22:36:44.331 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T22:36:44.387 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T22:36:44.401 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T22:36:44.401 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T22:36:44.402 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T22:36:44.402 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T22:36:44.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:36:44.407 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-21T22:36:44.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:36:44.408 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-21T22:36:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:5.15.0-56-generic 2024-09-21T22:36:44.455 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.15.0-56-generic 2024-09-21T22:36:44.455 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2024-09-21T22:36:44.456 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-56-generic 2024-09-21T22:36:44.456 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.15.0-56-generic 2024-09-21T22:36:44.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get clean 2024-09-21T22:36:44.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-21T22:36:44.588 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-09-21T22:36:44.716 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T22:36:44.828 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:36:44.833 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:36:44.846 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T22:36:44.897 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T22:36:44.915 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T22:36:44.945 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T22:36:45.037 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T22:36:45.074 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T22:36:45.077 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T22:36:45.077 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T22:36:45.077 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T22:36:45.096 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T22:36:45.099 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T22:36:45.099 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T22:36:45.103 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T22:36:45.109 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T22:36:45.111 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T22:36:45.111 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T22:36:45.111 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T22:36:45.111 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T22:36:45.127 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T22:36:45.253 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T22:36:45.271 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T22:36:45.290 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T22:36:45.391 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T22:36:45.409 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T22:36:45.585 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T22:36:45.586 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T22:36:45.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T22:36:45.747 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T22:36:45.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T22:36:45.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T22:36:45.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T22:36:45.765 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T22:36:45.773 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T22:36:45.773 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T22:36:45.779 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T22:36:45.779 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T22:36:45.781 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T22:36:45.830 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T22:36:45.835 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T22:36:45.850 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T22:36:45.870 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T22:36:45.870 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T22:36:45.870 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T22:36:45.870 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T22:36:45.870 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T22:36:45.871 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T22:36:45.877 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T22:36:45.886 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T22:36:45.886 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T22:36:45.888 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T22:36:45.897 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T22:36:45.898 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T22:36:45.904 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T22:36:45.910 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T22:36:45.910 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T22:36:45.918 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T22:36:45.919 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T22:36:45.924 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T22:36:45.924 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T22:36:45.925 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T22:36:45.926 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T22:36:45.929 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T22:36:45.930 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T22:36:45.931 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T22:36:45.981 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T22:36:45.981 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T22:36:45.982 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T22:36:45.982 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T22:36:45.983 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T22:36:45.991 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T22:36:45.991 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T22:36:45.992 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T22:36:46.005 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T22:36:46.006 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T22:36:46.006 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T22:36:46.007 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T22:36:46.009 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T22:36:46.010 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T22:36:46.011 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T22:36:46.011 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T22:36:46.012 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T22:36:46.074 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T22:36:46.074 INFO:teuthology.orchestra.run.smithi099.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T22:36:46.092 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T22:36:46.093 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T22:36:51.444 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 16.3 MB in 2s (7,524 kB/s) 2024-09-21T22:36:51.987 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 16.3 MB in 3s (5,862 kB/s) 2024-09-21T22:36:52.922 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:36:52.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:36:53.006 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:36:53.219 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-21T22:36:53.219 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-21T22:36:53.409 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:36:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:36:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:36:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:36:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T22:36:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: python2.7-minimal 2024-09-21T22:36:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:36:53.411 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-21T22:36:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:36:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T22:36:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T22:36:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-21T22:36:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T22:36:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-09-21T22:36:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: thermald 2024-09-21T22:36:53.464 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-21T22:36:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:36:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T22:36:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T22:36:53.466 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-09-21T22:36:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T22:36:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:36:53.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:36:53.642 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:36:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:36:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 113 MB of archives. 2024-09-21T22:36:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T22:36:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T22:36:53.850 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-21T22:36:53.851 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-21T22:36:54.026 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:36:54.026 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:36:54.026 INFO:teuthology.orchestra.run.smithi099.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:36:54.026 INFO:teuthology.orchestra.run.smithi099.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:36:54.026 INFO:teuthology.orchestra.run.smithi099.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T22:36:54.027 INFO:teuthology.orchestra.run.smithi099.stdout: python2.7-minimal 2024-09-21T22:36:54.027 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:36:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-21T22:36:54.028 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:36:54.028 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T22:36:54.028 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T22:36:54.029 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-21T22:36:54.030 INFO:teuthology.orchestra.run.smithi099.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T22:36:54.030 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2024-09-21T22:36:54.030 INFO:teuthology.orchestra.run.smithi099.stdout: thermald 2024-09-21T22:36:54.068 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-21T22:36:54.068 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:36:54.068 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T22:36:54.069 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T22:36:54.070 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2024-09-21T22:36:54.071 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T22:36:54.120 INFO:teuthology.orchestra.run.smithi099.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:36:54.121 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 113 MB of archives. 2024-09-21T22:36:54.121 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T22:36:54.121 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T22:36:54.568 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T22:36:54.685 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T22:36:55.041 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T22:36:55.444 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T22:36:55.524 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T22:36:55.525 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T22:36:55.525 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T22:36:55.649 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T22:36:55.671 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T22:36:56.063 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 113 MB in 2s (71.1 MB/s) 2024-09-21T22:36:56.233 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T22:36:57.586 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T22:36:57.587 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T22:36:57.587 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T22:36:58.057 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T22:36:58.169 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T22:36:58.238 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T22:36:58.242 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:36:58.304 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:36:58.553 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 113 MB in 5s (24.1 MB/s) 2024-09-21T22:36:58.755 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T22:37:00.621 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T22:37:00.643 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:00.799 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T22:37:00.804 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:00.843 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:00.893 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:01.291 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T22:37:01.311 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:01.349 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:03.226 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T22:37:03.247 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:03.840 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T22:37:03.861 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:03.899 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:07.229 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:07.356 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:07.721 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:07.843 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:08.129 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T22:37:08.151 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T22:37:08.187 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:37:10.509 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:10.637 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:10.978 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:11.099 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:11.386 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T22:37:11.409 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T22:37:11.452 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:37:14.139 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T22:37:14.175 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:14.214 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:37:17.841 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:18.835 INFO:teuthology.orchestra.run.smithi099.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:18.835 INFO:teuthology.orchestra.run.smithi099.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:18.836 INFO:teuthology.orchestra.run.smithi099.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:18.836 INFO:teuthology.orchestra.run.smithi099.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:18.994 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:21.127 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T22:37:21.253 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T22:37:21.284 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:37:21.379 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:21.410 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:37:21.761 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:37:21.907 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:22.034 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T22:37:22.163 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:22.756 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:22.806 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T22:37:22.807 INFO:teuthology.orchestra.run.smithi099.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:23.633 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:25.035 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T22:37:25.161 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T22:37:25.287 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:27.073 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:37:27.141 INFO:teuthology.orchestra.run.smithi094.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T22:37:27.142 INFO:teuthology.orchestra.run.smithi094.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T22:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub' 2024-09-21T22:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout:Generating grub configuration file ... 2024-09-21T22:37:37.659 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:37.676 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:37.982 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:37.988 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:38.060 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:37:38.065 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:37:38.210 INFO:teuthology.orchestra.run.smithi099.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:37:38.210 INFO:teuthology.orchestra.run.smithi099.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:37:38.210 INFO:teuthology.orchestra.run.smithi099.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:37:38.244 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-09-21T22:37:38.773 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 21 2024-09-21T22:37:38.795 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.15.0.122.122 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:37:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2024-09-21T22:37:38.796 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T22:37:38.797 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T22:37:38.797 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T22:37:38.797 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T22:37:38.797 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T22:37:38.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2024-09-21T22:37:38.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-21T22:37:39.068 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:37:39.189 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:37:39.281 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:37:39.351 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:37:40.451 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:37:40.468 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:37:40.522 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:37:40.713 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-21T22:37:40.714 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-21T22:37:40.885 INFO:teuthology.orchestra.run.smithi099.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:37:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:37:40.887 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:37:40.928 INFO:teuthology.orchestra.run.smithi099.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:37:40.929 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2024-09-21T22:37:40.988 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 21 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.15.0.122.122 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2024-09-21T22:37:40.989 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2024-09-21T22:37:40.990 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:37:40.990 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2024-09-21T22:37:40.990 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-21T22:37:41.034 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.F1tfkIqy14 2024-09-21T22:37:41.034 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub/grub.cfg /tmp/tmp.F1tfkIqy14 2024-09-21T22:37:41.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.F1tfkIqy14 2024-09-21T22:37:41.207 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.F1tfkIqy14 is 10KB 2024-09-21T22:37:41.220 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.F1tfkIqy14 2024-09-21T22:37:41.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T22:37:41.281 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:37:41.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T22:37:41.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T22:37:41.298 INFO:teuthology.orchestra.run.smithi094.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T22:37:41.298 INFO:teuthology.orchestra.run.smithi094.stdout:Sourcing file `/etc/default/grub' 2024-09-21T22:37:41.313 INFO:teuthology.orchestra.run.smithi094.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:37:41.330 INFO:teuthology.orchestra.run.smithi094.stdout:Generating grub configuration file ... 2024-09-21T22:37:41.355 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T22:37:41.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo update-grub 2024-09-21T22:37:41.594 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:41.618 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:41.922 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:41.927 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:41.993 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:37:41.999 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:37:42.166 INFO:teuthology.orchestra.run.smithi094.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:37:42.166 INFO:teuthology.orchestra.run.smithi094.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:37:42.167 INFO:teuthology.orchestra.run.smithi094.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:37:42.218 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-09-21T22:37:42.523 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub' 2024-09-21T22:37:42.524 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:37:42.526 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-21T22:37:42.699 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:42.702 DEBUG:teuthology.orchestra.run.smithi094:> dpkg -s linux-image-generic 2024-09-21T22:37:42.705 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:42.729 INFO:teuthology.orchestra.run.smithi094.stdout:Package: linux-image-generic 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Status: install ok installed 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Priority: optional 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Section: kernel 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Installed-Size: 21 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: amd64 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Source: linux-meta 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Version: 5.15.0.122.122 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:37:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Recommends: thermald 2024-09-21T22:37:42.731 INFO:teuthology.orchestra.run.smithi094.stdout:Description: Generic Linux kernel image 2024-09-21T22:37:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:37:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: available. 2024-09-21T22:37:42.731 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T22:37:42.731 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T22:37:42.731 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T22:37:42.731 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T22:37:42.731 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T22:37:42.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get clean 2024-09-21T22:37:42.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-09-21T22:37:42.933 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:42.939 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:42.976 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:37:42.981 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:37:43.036 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:37:43.117 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:37:43.117 INFO:teuthology.orchestra.run.smithi099.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:37:43.117 INFO:teuthology.orchestra.run.smithi099.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:37:43.134 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-21T22:37:43.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-09-21T22:37:43.158 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:37:43.239 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:37:43.321 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:37:44.496 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:37:44.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:37:44.576 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:37:44.790 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-21T22:37:44.791 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-21T22:37:44.975 INFO:teuthology.orchestra.run.smithi094.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T22:37:44.975 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:37:44.976 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:37:45.016 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:37:45.018 DEBUG:teuthology.orchestra.run.smithi094:> dpkg -s linux-image-generic 2024-09-21T22:37:45.029 INFO:teuthology.orchestra.run.smithi094.stdout:Package: linux-image-generic 2024-09-21T22:37:45.029 INFO:teuthology.orchestra.run.smithi094.stdout:Status: install ok installed 2024-09-21T22:37:45.029 INFO:teuthology.orchestra.run.smithi094.stdout:Priority: optional 2024-09-21T22:37:45.029 INFO:teuthology.orchestra.run.smithi094.stdout:Section: kernel 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Installed-Size: 21 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: amd64 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Source: linux-meta 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Version: 5.15.0.122.122 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Recommends: thermald 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout:Description: Generic Linux kernel image 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:37:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: available. 2024-09-21T22:37:45.031 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-21T22:37:45.078 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.rcz8Yf32dt 2024-09-21T22:37:45.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rcz8Yf32dt 2024-09-21T22:37:45.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.rcz8Yf32dt 2024-09-21T22:37:45.260 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.rcz8Yf32dt is 10KB 2024-09-21T22:37:45.272 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.rcz8Yf32dt 2024-09-21T22:37:45.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T22:37:45.330 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:37:45.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T22:37:45.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T22:37:45.403 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T22:37:45.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo update-grub 2024-09-21T22:37:46.728 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-09-21T22:37:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:37:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-21T22:37:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:37:46.908 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:37:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:37:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:37:47.168 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:37:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:37:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:37:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:37:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:37:47.318 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-21T22:37:47.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-09-21T22:38:13.170 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T22:38:13.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:38:13.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:38:17.330 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T22:38:17.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:38:17.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:38:31.667 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-21T22:38:35.759 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-21T22:38:40.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:38:40.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:38:43.727 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-21T22:38:44.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:38:44.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:38:47.823 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-21T22:38:55.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:38:55.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:38:59.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:38:59.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:39:14.159 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-21T22:39:18.255 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-21T22:39:29.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:39:29.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:39:29.624 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-21T22:39:30.542 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:39:30.542 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T22:39:30.542 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-21T22:39:30.590 INFO:teuthology.orchestra.run.smithi099.stdout:5.15.0-122-generic 2024-09-21T22:39:30.590 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T22:39:30.590 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T22:39:30.590 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T22:39:31.592 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T22:39:31.592 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T22:39:31.660 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-09-21T22:39:31.679 DEBUG:teuthology.parallel:result is None 2024-09-21T22:39:33.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:39:33.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:39:33.515 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-21T22:39:34.124 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:39:34.124 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T22:39:34.124 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-21T22:39:34.172 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-122-generic 2024-09-21T22:39:34.172 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T22:39:34.172 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T22:39:34.172 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T22:39:35.173 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T22:39:35.174 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T22:39:35.246 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-09-21T22:39:35.261 DEBUG:teuthology.parallel:result is None 2024-09-21T22:39:35.262 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T22:39:35.276 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T22:39:35.276 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T22:39:35.278 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T22:39:35.284 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T22:39:35.362 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T22:39:35.371 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T22:39:35.371 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T22:39:35.374 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T22:39:35.390 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T22:39:35.398 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T22:39:35.398 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T22:39:35.425 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T22:39:35.446 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:39:35.454 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:39:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:39:35.467 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:39:35.468 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T22:39:35.475 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T22:39:35.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T22:39:35.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T22:39:35.522 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T22:39:35.529 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T22:39:35.530 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T22:39:35.557 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T22:39:35.570 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T22:39:35.609 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T22:39:35.657 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:39:35.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T22:39:35.725 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T22:39:35.730 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T22:39:35.778 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:39:35.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T22:39:35.867 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-21T22:39:35.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-21T22:39:35.907 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T22:39:35.914 INFO:teuthology.task.internal:Starting timer... 2024-09-21T22:39:35.914 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T22:39:35.925 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T22:39:35.935 DEBUG:teuthology.task.selinux:Excluding smithi094: OS 'ubuntu' does not support SELinux 2024-09-21T22:39:35.935 DEBUG:teuthology.task.selinux:Excluding smithi099: OS 'ubuntu' does not support SELinux 2024-09-21T22:39:35.935 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T22:39:35.936 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T22:39:35.936 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T22:39:35.936 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T22:39:35.950 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T22:39:35.960 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T22:39:35.960 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T22:47:02.914 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com')] 2024-09-21T22:47:02.915 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:47:02.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:47:02.993 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-21T22:47:03.074 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-21T22:47:03.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:47:03.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:47:03.154 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-21T22:47:03.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T22:47:03.231 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T22:47:03.239 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T22:47:03.240 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T22:47:03.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T22:47:03.242 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T22:47:03.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Command line: ntpd -gq 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: ---------------------------------------------------- 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: corporation. Support and training for ntp-4 are 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: available at https://www.nwtime.org/support 2024-09-21T22:47:03.270 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: ---------------------------------------------------- 2024-09-21T22:47:03.271 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: proto: precision = 0.059 usec (-24) 2024-09-21T22:47:03.271 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: basedate set to 2022-02-04 2024-09-21T22:47:03.271 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: gps base set to 2022-02-06 (week 2196) 2024-09-21T22:47:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T22:47:03.272 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T22:47:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen normally on 3 enp3s0f1 172.21.15.94:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen normally on 4 lo [::1]:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d763%5]:123 2024-09-21T22:47:03.273 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:03 ntpd[18486]: Listening on routing socket on fd #22 for interface updates 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Command line: ntpd -gq 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: ---------------------------------------------------- 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: corporation. Support and training for ntp-4 are 2024-09-21T22:47:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: available at https://www.nwtime.org/support 2024-09-21T22:47:03.300 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: ---------------------------------------------------- 2024-09-21T22:47:03.300 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: proto: precision = 0.045 usec (-24) 2024-09-21T22:47:03.300 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: basedate set to 2022-02-04 2024-09-21T22:47:03.300 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: gps base set to 2022-02-06 (week 2196) 2024-09-21T22:47:03.301 INFO:teuthology.orchestra.run.smithi099.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T22:47:03.301 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T22:47:03.301 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T22:47:03.301 INFO:teuthology.orchestra.run.smithi099.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen normally on 3 enp3s0f1 172.21.15.99:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen normally on 4 lo [::1]:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d803%5]:123 2024-09-21T22:47:03.302 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:03 ntpd[18457]: Listening on routing socket on fd #22 for interface updates 2024-09-21T22:47:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:04 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:05.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:05 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:06.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:06.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:06 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:06 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:07.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:07.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:07 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:08 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:08 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:09 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:10.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:10 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:10.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.300 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stdout:21 Sep 22:47:10 ntpd[18457]: ntpd: time slew -0.002504 s 2024-09-21T22:47:10.301 INFO:teuthology.orchestra.run.smithi099.stdout:ntpd: time slew -0.002504s 2024-09-21T22:47:10.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T22:47:10.302 INFO:teuthology.orchestra.run.smithi099.stderr:21 Sep 22:47:10 ntpd[18457]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T22:47:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T22:47:10.352 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-21T22:47:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:10.352 INFO:teuthology.orchestra.run.smithi099.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:11.270 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stdout:21 Sep 22:47:11 ntpd[18486]: ntpd: time slew -0.002291 s 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stdout:ntpd: time slew -0.002291s 2024-09-21T22:47:11.271 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T22:47:11.272 INFO:teuthology.orchestra.run.smithi094.stderr:21 Sep 22:47:11 ntpd[18486]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:47:11.324 INFO:teuthology.run_tasks:Running task install... 2024-09-21T22:47:11.332 DEBUG:teuthology.task.install:project ceph 2024-09-21T22:47:11.332 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T22:47:11.332 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-21T22:47:11.332 INFO:teuthology.task.install:Using flavor: default 2024-09-21T22:47:11.338 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T22:47:11.338 INFO:teuthology.task.install:extra packages: [] 2024-09-21T22:47:11.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-key list | grep Ceph 2024-09-21T22:47:11.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-key list | grep Ceph 2024-09-21T22:47:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T22:47:11.436 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T22:47:11.437 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T22:47:11.438 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T22:47:11.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:47:11.442 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T22:47:11.471 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T22:47:11.471 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T22:47:11.472 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T22:47:11.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:47:11.578 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T22:47:11.578 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:47:11.612 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T22:47:11.612 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:47:11.692 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:47:11.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T22:47:11.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-21T22:47:11.723 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:47:11.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T22:47:11.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-09-21T22:47:11.877 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:47:11.877 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:47:11.888 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:47:11.901 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:47:11.905 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:47:11.911 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:47:11.929 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:47:11.932 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T22:47:11.954 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T22:47:11.955 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T22:47:11.976 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T22:47:11.976 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T22:47:11.997 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T22:47:12.003 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T22:47:12.025 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T22:47:12.025 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T22:47:12.025 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:47:12.051 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T22:47:12.218 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 28.9 kB in 0s (87.5 kB/s) 2024-09-21T22:47:12.258 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 28.9 kB in 0s (85.0 kB/s) 2024-09-21T22:47:13.243 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:47:13.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:47:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:47:13.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:47:13.322 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-21T22:47:13.380 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-21T22:47:13.533 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-21T22:47:13.534 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-21T22:47:13.589 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-21T22:47:13.590 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-21T22:47:13.731 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:47:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:47:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:47:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:47:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:47:13.732 INFO:teuthology.orchestra.run.smithi099.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:47:13.732 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:47:13.732 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-21T22:47:13.732 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T22:47:13.732 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T22:47:13.733 INFO:teuthology.orchestra.run.smithi099.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T22:47:13.733 INFO:teuthology.orchestra.run.smithi099.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T22:47:13.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet zip 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-21T22:47:13.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2024-09-21T22:47:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: btrfs-tools 2024-09-21T22:47:13.797 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:47:13.798 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:47:13.799 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-21T22:47:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T22:47:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T22:47:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T22:47:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T22:47:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet zip 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-21T22:47:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-09-21T22:47:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: btrfs-tools 2024-09-21T22:47:13.837 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-21T22:47:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T22:47:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T22:47:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T22:47:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T22:47:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T22:47:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T22:47:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T22:47:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-21T22:47:13.904 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T22:47:13.905 INFO:teuthology.orchestra.run.smithi094.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T22:47:13.906 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T22:47:13.907 INFO:teuthology.orchestra.run.smithi094.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-21T22:47:13.982 INFO:teuthology.orchestra.run.smithi099.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T22:47:13.982 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 181 MB of archives. 2024-09-21T22:47:13.982 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T22:47:13.982 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T22:47:14.027 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T22:47:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T22:47:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 181 MB of archives. 2024-09-21T22:47:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T22:47:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T22:47:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T22:47:14.199 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T22:47:14.266 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T22:47:14.287 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T22:47:14.305 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T22:47:14.313 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T22:47:14.314 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T22:47:14.317 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T22:47:14.328 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T22:47:14.334 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T22:47:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T22:47:14.402 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T22:47:14.410 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T22:47:14.411 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T22:47:14.415 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T22:47:14.417 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T22:47:14.436 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T22:47:14.450 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T22:47:14.462 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T22:47:14.517 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T22:47:14.529 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T22:47:14.542 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T22:47:14.543 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T22:47:14.545 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T22:47:14.564 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T22:47:14.611 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T22:47:14.678 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T22:47:14.681 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T22:47:14.690 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T22:47:14.772 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T22:47:14.783 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T22:47:14.813 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T22:47:14.821 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T22:47:14.822 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T22:47:14.824 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T22:47:14.840 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T22:47:14.841 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T22:47:14.846 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T22:47:14.850 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T22:47:14.850 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T22:47:14.854 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T22:47:14.881 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T22:47:14.889 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T22:47:14.890 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T22:47:14.891 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T22:47:14.909 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T22:47:14.914 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T22:47:14.918 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T22:47:14.922 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T22:47:14.934 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T22:47:14.938 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T22:47:14.942 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T22:47:14.946 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T22:47:14.947 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T22:47:14.947 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T22:47:14.949 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T22:47:14.950 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T22:47:14.951 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T22:47:15.002 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T22:47:15.007 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T22:47:15.011 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T22:47:15.014 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T22:47:15.015 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T22:47:15.016 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T22:47:15.017 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T22:47:15.018 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T22:47:15.018 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T22:47:15.019 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T22:47:15.033 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T22:47:15.034 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T22:47:15.034 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T22:47:15.039 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T22:47:15.040 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T22:47:15.085 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T22:47:15.099 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T22:47:15.099 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T22:47:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T22:47:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T22:47:15.102 INFO:teuthology.orchestra.run.smithi099.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T22:47:15.102 INFO:teuthology.orchestra.run.smithi099.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T22:47:15.103 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T22:47:15.108 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T22:47:15.109 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T22:47:15.167 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T22:47:15.168 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T22:47:15.170 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T22:47:15.170 INFO:teuthology.orchestra.run.smithi094.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T22:47:15.180 INFO:teuthology.orchestra.run.smithi099.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T22:47:15.182 INFO:teuthology.orchestra.run.smithi099.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T22:47:15.183 INFO:teuthology.orchestra.run.smithi099.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T22:47:15.184 INFO:teuthology.orchestra.run.smithi099.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T22:47:15.195 INFO:teuthology.orchestra.run.smithi099.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T22:47:15.248 INFO:teuthology.orchestra.run.smithi094.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T22:47:15.251 INFO:teuthology.orchestra.run.smithi094.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T22:47:15.252 INFO:teuthology.orchestra.run.smithi094.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T22:47:15.253 INFO:teuthology.orchestra.run.smithi094.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T22:47:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T22:47:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T22:47:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T22:47:15.263 INFO:teuthology.orchestra.run.smithi094.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T22:47:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T22:47:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T22:47:15.330 INFO:teuthology.orchestra.run.smithi094.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T22:47:15.331 INFO:teuthology.orchestra.run.smithi094.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T22:47:15.332 INFO:teuthology.orchestra.run.smithi094.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T22:47:15.334 INFO:teuthology.orchestra.run.smithi094.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T22:47:15.338 INFO:teuthology.orchestra.run.smithi094.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T22:47:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T22:47:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T22:47:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T22:47:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T22:47:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T22:47:15.411 INFO:teuthology.orchestra.run.smithi094.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T22:47:15.414 INFO:teuthology.orchestra.run.smithi094.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T22:47:15.415 INFO:teuthology.orchestra.run.smithi094.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T22:47:15.420 INFO:teuthology.orchestra.run.smithi094.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T22:47:15.421 INFO:teuthology.orchestra.run.smithi094.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T22:47:15.424 INFO:teuthology.orchestra.run.smithi099.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T22:47:15.481 INFO:teuthology.orchestra.run.smithi099.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T22:47:15.481 INFO:teuthology.orchestra.run.smithi099.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T22:47:15.492 INFO:teuthology.orchestra.run.smithi094.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T22:47:15.497 INFO:teuthology.orchestra.run.smithi094.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T22:47:15.593 INFO:teuthology.orchestra.run.smithi099.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T22:47:15.597 INFO:teuthology.orchestra.run.smithi099.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T22:47:15.600 INFO:teuthology.orchestra.run.smithi099.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T22:47:15.609 INFO:teuthology.orchestra.run.smithi094.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T22:47:15.637 INFO:teuthology.orchestra.run.smithi099.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T22:47:15.638 INFO:teuthology.orchestra.run.smithi099.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T22:47:15.640 INFO:teuthology.orchestra.run.smithi099.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T22:47:15.641 INFO:teuthology.orchestra.run.smithi099.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T22:47:15.643 INFO:teuthology.orchestra.run.smithi099.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T22:47:15.662 INFO:teuthology.orchestra.run.smithi094.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T22:47:15.664 INFO:teuthology.orchestra.run.smithi094.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T22:47:15.680 INFO:teuthology.orchestra.run.smithi099.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T22:47:15.684 INFO:teuthology.orchestra.run.smithi099.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T22:47:15.684 INFO:teuthology.orchestra.run.smithi099.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T22:47:15.696 INFO:teuthology.orchestra.run.smithi094.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T22:47:15.697 INFO:teuthology.orchestra.run.smithi094.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T22:47:15.699 INFO:teuthology.orchestra.run.smithi094.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T22:47:15.706 INFO:teuthology.orchestra.run.smithi099.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T22:47:15.732 INFO:teuthology.orchestra.run.smithi094.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T22:47:15.736 INFO:teuthology.orchestra.run.smithi094.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T22:47:15.738 INFO:teuthology.orchestra.run.smithi094.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T22:47:15.741 INFO:teuthology.orchestra.run.smithi094.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T22:47:15.745 INFO:teuthology.orchestra.run.smithi094.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T22:47:15.745 INFO:teuthology.orchestra.run.smithi094.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T22:47:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T22:47:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T22:47:15.777 INFO:teuthology.orchestra.run.smithi094.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T22:47:15.802 INFO:teuthology.orchestra.run.smithi099.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T22:47:15.803 INFO:teuthology.orchestra.run.smithi099.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T22:47:15.804 INFO:teuthology.orchestra.run.smithi099.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T22:47:15.805 INFO:teuthology.orchestra.run.smithi099.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T22:47:15.806 INFO:teuthology.orchestra.run.smithi099.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T22:47:15.807 INFO:teuthology.orchestra.run.smithi099.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T22:47:15.814 INFO:teuthology.orchestra.run.smithi099.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T22:47:15.824 INFO:teuthology.orchestra.run.smithi094.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T22:47:15.824 INFO:teuthology.orchestra.run.smithi094.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T22:47:15.839 INFO:teuthology.orchestra.run.smithi099.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T22:47:15.860 INFO:teuthology.orchestra.run.smithi094.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T22:47:15.861 INFO:teuthology.orchestra.run.smithi094.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T22:47:15.862 INFO:teuthology.orchestra.run.smithi094.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T22:47:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T22:47:15.864 INFO:teuthology.orchestra.run.smithi094.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T22:47:15.865 INFO:teuthology.orchestra.run.smithi094.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T22:47:15.872 INFO:teuthology.orchestra.run.smithi094.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T22:47:15.884 INFO:teuthology.orchestra.run.smithi099.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T22:47:15.886 INFO:teuthology.orchestra.run.smithi099.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T22:47:15.888 INFO:teuthology.orchestra.run.smithi099.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T22:47:15.941 INFO:teuthology.orchestra.run.smithi094.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T22:47:15.945 INFO:teuthology.orchestra.run.smithi094.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T22:47:15.946 INFO:teuthology.orchestra.run.smithi094.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T22:47:15.965 INFO:teuthology.orchestra.run.smithi099.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T22:47:15.967 INFO:teuthology.orchestra.run.smithi094.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T22:47:15.971 INFO:teuthology.orchestra.run.smithi099.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T22:47:15.973 INFO:teuthology.orchestra.run.smithi099.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T22:47:15.974 INFO:teuthology.orchestra.run.smithi099.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T22:47:15.986 INFO:teuthology.orchestra.run.smithi099.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T22:47:15.987 INFO:teuthology.orchestra.run.smithi099.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T22:47:15.995 INFO:teuthology.orchestra.run.smithi099.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T22:47:16.023 INFO:teuthology.orchestra.run.smithi094.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T22:47:16.029 INFO:teuthology.orchestra.run.smithi094.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T22:47:16.031 INFO:teuthology.orchestra.run.smithi094.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T22:47:16.032 INFO:teuthology.orchestra.run.smithi094.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T22:47:16.044 INFO:teuthology.orchestra.run.smithi094.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T22:47:16.045 INFO:teuthology.orchestra.run.smithi094.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T22:47:16.046 INFO:teuthology.orchestra.run.smithi099.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T22:47:16.048 INFO:teuthology.orchestra.run.smithi099.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T22:47:16.048 INFO:teuthology.orchestra.run.smithi099.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T22:47:16.053 INFO:teuthology.orchestra.run.smithi094.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T22:47:16.104 INFO:teuthology.orchestra.run.smithi094.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T22:47:16.106 INFO:teuthology.orchestra.run.smithi094.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T22:47:16.106 INFO:teuthology.orchestra.run.smithi094.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T22:47:16.128 INFO:teuthology.orchestra.run.smithi099.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T22:47:16.129 INFO:teuthology.orchestra.run.smithi099.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T22:47:16.146 INFO:teuthology.orchestra.run.smithi099.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T22:47:16.147 INFO:teuthology.orchestra.run.smithi099.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T22:47:16.148 INFO:teuthology.orchestra.run.smithi099.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T22:47:16.153 INFO:teuthology.orchestra.run.smithi099.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T22:47:16.155 INFO:teuthology.orchestra.run.smithi099.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T22:47:16.185 INFO:teuthology.orchestra.run.smithi094.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T22:47:16.188 INFO:teuthology.orchestra.run.smithi094.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T22:47:16.206 INFO:teuthology.orchestra.run.smithi094.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T22:47:16.207 INFO:teuthology.orchestra.run.smithi094.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T22:47:16.207 INFO:teuthology.orchestra.run.smithi094.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T22:47:16.213 INFO:teuthology.orchestra.run.smithi094.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T22:47:16.214 INFO:teuthology.orchestra.run.smithi094.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T22:47:16.736 INFO:teuthology.orchestra.run.smithi099.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T22:47:16.736 INFO:teuthology.orchestra.run.smithi099.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T22:47:16.755 INFO:teuthology.orchestra.run.smithi099.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T22:47:16.773 INFO:teuthology.orchestra.run.smithi094.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T22:47:16.774 INFO:teuthology.orchestra.run.smithi094.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T22:47:16.791 INFO:teuthology.orchestra.run.smithi094.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T22:47:16.798 INFO:teuthology.orchestra.run.smithi099.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T22:47:16.808 INFO:teuthology.orchestra.run.smithi099.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T22:47:16.810 INFO:teuthology.orchestra.run.smithi099.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T22:47:16.833 INFO:teuthology.orchestra.run.smithi094.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T22:47:16.842 INFO:teuthology.orchestra.run.smithi094.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T22:47:16.844 INFO:teuthology.orchestra.run.smithi094.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T22:47:16.846 INFO:teuthology.orchestra.run.smithi099.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T22:47:16.896 INFO:teuthology.orchestra.run.smithi094.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T22:47:17.143 INFO:teuthology.orchestra.run.smithi099.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T22:47:17.144 INFO:teuthology.orchestra.run.smithi099.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T22:47:17.175 INFO:teuthology.orchestra.run.smithi094.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T22:47:17.175 INFO:teuthology.orchestra.run.smithi094.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T22:47:19.093 INFO:teuthology.orchestra.run.smithi094.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T22:47:19.095 INFO:teuthology.orchestra.run.smithi094.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T22:47:19.096 INFO:teuthology.orchestra.run.smithi094.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T22:47:19.243 INFO:teuthology.orchestra.run.smithi099.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T22:47:19.245 INFO:teuthology.orchestra.run.smithi099.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T22:47:19.247 INFO:teuthology.orchestra.run.smithi099.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T22:47:19.290 INFO:teuthology.orchestra.run.smithi094.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T22:47:19.447 INFO:teuthology.orchestra.run.smithi099.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T22:47:19.860 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 181 MB in 5s (33.7 MB/s) 2024-09-21T22:47:19.992 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T22:47:19.994 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 181 MB in 6s (32.3 MB/s) 2024-09-21T22:47:20.082 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T22:47:20.088 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T22:47:20.125 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T22:47:20.157 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T22:47:20.204 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T22:47:20.209 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T22:47:20.300 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T22:47:20.478 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T22:47:20.504 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T22:47:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T22:47:20.636 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T22:47:20.663 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T22:47:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T22:47:20.914 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T22:47:20.941 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T22:47:20.981 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T22:47:21.048 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T22:47:21.075 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T22:47:21.115 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T22:47:21.385 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T22:47:21.411 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:21.468 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:21.493 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T22:47:21.520 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:21.559 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:21.905 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T22:47:21.930 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T22:47:21.931 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:21.956 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:21.971 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:21.996 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:22.325 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T22:47:22.333 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T22:47:22.351 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:22.359 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T22:47:22.391 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:22.400 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:47:22.695 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T22:47:22.721 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T22:47:22.728 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T22:47:22.753 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T22:47:22.754 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T22:47:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T22:47:23.056 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librados2. 2024-09-21T22:47:23.064 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librados2. 2024-09-21T22:47:23.082 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:23.125 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:23.131 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:23.592 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librbd1. 2024-09-21T22:47:23.600 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librbd1. 2024-09-21T22:47:23.618 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:23.627 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:23.667 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:23.684 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.095 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T22:47:24.120 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T22:47:24.121 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:24.146 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:24.153 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.186 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.422 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rados. 2024-09-21T22:47:24.437 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rados. 2024-09-21T22:47:24.472 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.483 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:24.522 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.792 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T22:47:24.818 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:24.851 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T22:47:24.858 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:24.877 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:24.917 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:25.170 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T22:47:25.185 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:25.228 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:25.345 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T22:47:25.371 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:25.411 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:25.506 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T22:47:25.522 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:25.556 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:25.697 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T22:47:25.724 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:25.758 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:25.868 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T22:47:25.894 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T22:47:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T22:47:26.069 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T22:47:26.096 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T22:47:26.136 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T22:47:26.262 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T22:47:26.289 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T22:47:26.329 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T22:47:26.455 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T22:47:26.481 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T22:47:26.522 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T22:47:26.631 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T22:47:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:26.690 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T22:47:26.798 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:26.833 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:27.035 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T22:47:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T22:47:27.101 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T22:47:27.136 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T22:47:27.162 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T22:47:27.202 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T22:47:27.346 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T22:47:27.372 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T22:47:27.412 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T22:47:27.455 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T22:47:27.470 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T22:47:27.513 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T22:47:27.716 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T22:47:27.731 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T22:47:27.774 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T22:47:27.817 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T22:47:27.843 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T22:47:27.883 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T22:47:28.144 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua5.1. 2024-09-21T22:47:28.170 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T22:47:28.211 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T22:47:28.211 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua5.1. 2024-09-21T22:47:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T22:47:28.278 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T22:47:28.489 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-any. 2024-09-21T22:47:28.504 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T22:47:28.547 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T22:47:28.556 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-any. 2024-09-21T22:47:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T22:47:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T22:47:28.800 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package zip. 2024-09-21T22:47:28.815 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T22:47:28.858 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T22:47:28.861 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package zip. 2024-09-21T22:47:28.886 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T22:47:28.926 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T22:47:29.178 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package unzip. 2024-09-21T22:47:29.204 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T22:47:29.229 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package unzip. 2024-09-21T22:47:29.245 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T22:47:29.256 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T22:47:29.295 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T22:47:29.624 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package luarocks. 2024-09-21T22:47:29.632 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package luarocks. 2024-09-21T22:47:29.639 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T22:47:29.648 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T22:47:29.674 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T22:47:29.683 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T22:47:30.024 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librgw2. 2024-09-21T22:47:30.049 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librgw2. 2024-09-21T22:47:30.051 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:30.075 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:30.090 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:30.115 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:30.651 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T22:47:30.677 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:30.718 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:30.819 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T22:47:30.834 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:30.877 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:31.063 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T22:47:31.089 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T22:47:31.129 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T22:47:31.197 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T22:47:31.223 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T22:47:31.255 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T22:47:31.441 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T22:47:31.467 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:31.507 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:31.533 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T22:47:31.559 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:31.599 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:32.028 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-common. 2024-09-21T22:47:32.054 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:32.094 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:32.244 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-common. 2024-09-21T22:47:32.271 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:32.311 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:33.417 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-base. 2024-09-21T22:47:33.443 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:33.650 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-base. 2024-09-21T22:47:33.676 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:33.734 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:34.038 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T22:47:34.065 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T22:47:34.104 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T22:47:34.238 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T22:47:34.264 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T22:47:34.304 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T22:47:34.407 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T22:47:34.433 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T22:47:34.474 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T22:47:34.616 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T22:47:34.642 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T22:47:34.682 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T22:47:34.794 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T22:47:34.820 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T22:47:34.860 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T22:47:35.019 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T22:47:35.045 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T22:47:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T22:47:35.138 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T22:47:35.165 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T22:47:35.205 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T22:47:35.389 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T22:47:35.415 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T22:47:35.455 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T22:47:35.475 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T22:47:35.501 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T22:47:35.550 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T22:47:35.750 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T22:47:35.776 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T22:47:35.816 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T22:47:36.061 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T22:47:36.088 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T22:47:36.119 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T22:47:36.128 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T22:47:36.146 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T22:47:36.186 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T22:47:36.430 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-portend. 2024-09-21T22:47:36.456 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-portend. 2024-09-21T22:47:36.457 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T22:47:36.483 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T22:47:36.497 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T22:47:36.522 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T22:47:36.800 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T22:47:36.826 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T22:47:36.826 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T22:47:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T22:47:36.867 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T22:47:36.892 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T22:47:37.170 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T22:47:37.195 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T22:47:37.196 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T22:47:37.222 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T22:47:37.253 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T22:47:37.262 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T22:47:37.540 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T22:47:37.567 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T22:47:37.598 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T22:47:37.624 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T22:47:37.651 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T22:47:37.691 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T22:47:37.910 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T22:47:37.936 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T22:47:37.977 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T22:47:38.002 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T22:47:38.029 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T22:47:38.069 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T22:47:38.280 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T22:47:38.306 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T22:47:38.346 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T22:47:38.364 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T22:47:38.389 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T22:47:38.430 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T22:47:38.624 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-mako. 2024-09-21T22:47:38.650 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T22:47:38.682 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T22:47:38.733 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-mako. 2024-09-21T22:47:38.760 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T22:47:38.800 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T22:47:39.002 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T22:47:39.029 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T22:47:39.069 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T22:47:39.128 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T22:47:39.155 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T22:47:39.195 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T22:47:39.347 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T22:47:39.374 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T22:47:39.405 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T22:47:39.498 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T22:47:39.524 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T22:47:39.573 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T22:47:39.692 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webob. 2024-09-21T22:47:39.718 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T22:47:39.758 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T22:47:39.850 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webob. 2024-09-21T22:47:39.866 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T22:47:39.900 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T22:47:40.103 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T22:47:40.129 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T22:47:40.172 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T22:47:40.195 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T22:47:40.222 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T22:47:40.257 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T22:47:40.481 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T22:47:40.507 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T22:47:40.540 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T22:47:40.566 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T22:47:40.607 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T22:47:40.620 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T22:47:40.910 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-paste. 2024-09-21T22:47:40.936 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T22:47:40.959 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-paste. 2024-09-21T22:47:40.974 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T22:47:40.976 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T22:47:41.009 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T22:47:41.288 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T22:47:41.303 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T22:47:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T22:47:41.339 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T22:47:41.346 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T22:47:41.379 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T22:47:41.648 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T22:47:41.658 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T22:47:41.675 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T22:47:41.684 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T22:47:41.706 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T22:47:41.724 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T22:47:41.985 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T22:47:42.011 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T22:47:42.027 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T22:47:42.051 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T22:47:42.054 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T22:47:42.094 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T22:47:42.363 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T22:47:42.390 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T22:47:42.429 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T22:47:42.672 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T22:47:42.699 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T22:47:42.730 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T22:47:42.793 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T22:47:42.819 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T22:47:42.859 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T22:47:43.044 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T22:47:43.071 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T22:47:43.110 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T22:47:43.196 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-idna. 2024-09-21T22:47:43.223 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T22:47:43.262 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T22:47:43.438 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-idna. 2024-09-21T22:47:43.454 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T22:47:43.496 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T22:47:43.557 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T22:47:43.584 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T22:47:43.623 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T22:47:43.825 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T22:47:43.852 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T22:47:43.891 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T22:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests. 2024-09-21T22:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T22:47:44.027 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T22:47:44.203 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests. 2024-09-21T22:47:44.230 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T22:47:44.270 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T22:47:44.338 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T22:47:44.365 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T22:47:44.405 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T22:47:44.581 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T22:47:44.608 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T22:47:44.648 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T22:47:44.691 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T22:47:44.718 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:44.766 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:44.918 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T22:47:44.945 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:44.984 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:45.171 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T22:47:45.198 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:45.238 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:45.339 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T22:47:45.366 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:45.406 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:45.549 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T22:47:45.576 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T22:47:45.615 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T22:47:45.684 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T22:47:45.710 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T22:47:45.775 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T22:47:45.919 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T22:47:45.945 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T22:47:45.985 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T22:47:46.045 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T22:47:46.061 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T22:47:46.095 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T22:47:46.440 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T22:47:46.466 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:46.910 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T22:47:46.937 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:46.976 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:47.331 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T22:47:47.358 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:47.389 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:47.809 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T22:47:47.836 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:47.875 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:48.481 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T22:47:48.488 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T22:47:48.510 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:48.514 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:48.547 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:48.554 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:49.624 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph. 2024-09-21T22:47:49.639 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:49.644 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph. 2024-09-21T22:47:49.660 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:49.674 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:49.703 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:49.972 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T22:47:49.999 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.002 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T22:47:50.029 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.030 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:50.068 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:50.376 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T22:47:50.402 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.422 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T22:47:50.448 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.450 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:50.488 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:50.850 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephadm. 2024-09-21T22:47:50.862 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephadm. 2024-09-21T22:47:50.876 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.889 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:47:50.916 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:50.929 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:51.228 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T22:47:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T22:47:51.254 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T22:47:51.275 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T22:47:51.294 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T22:47:51.315 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T22:47:51.605 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T22:47:51.620 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T22:47:51.651 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T22:47:51.663 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T22:47:51.667 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T22:47:51.709 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T22:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-babel. 2024-09-21T22:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T22:47:52.710 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T22:47:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-babel. 2024-09-21T22:47:52.825 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T22:47:52.864 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T22:47:53.030 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T22:47:53.057 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T22:47:53.096 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T22:47:53.210 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T22:47:53.236 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T22:47:53.276 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T22:47:53.450 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T22:47:53.477 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:53.516 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:53.638 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T22:47:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:53.704 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:53.803 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T22:47:53.818 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T22:47:53.878 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T22:47:54.041 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T22:47:54.068 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T22:47:54.108 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T22:47:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-routes. 2024-09-21T22:47:54.184 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T22:47:54.223 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T22:47:54.386 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-routes. 2024-09-21T22:47:54.412 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T22:47:54.446 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T22:47:54.542 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T22:47:54.558 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:54.600 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:54.764 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T22:47:54.791 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:54.830 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:55.856 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T22:47:55.883 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T22:47:55.922 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T22:47:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T22:47:56.180 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T22:47:56.219 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T22:47:56.426 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T22:47:56.453 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T22:47:56.492 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T22:47:56.740 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T22:47:56.766 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T22:47:56.798 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T22:47:56.979 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T22:47:57.003 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T22:47:57.037 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T22:47:57.202 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T22:47:57.229 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T22:47:57.268 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T22:47:57.340 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T22:47:57.367 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T22:47:57.407 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T22:47:57.580 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T22:47:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T22:47:57.646 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T22:47:58.133 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T22:47:58.160 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:58.293 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:58.555 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T22:47:58.583 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:47:58.715 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:47:59.259 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T22:47:59.282 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T22:47:59.321 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T22:47:59.625 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T22:47:59.652 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T22:47:59.653 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T22:47:59.680 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T22:47:59.691 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T22:47:59.719 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T22:47:59.956 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T22:47:59.983 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T22:48:00.013 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T22:48:00.028 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T22:48:00.055 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T22:48:00.095 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T22:48:00.367 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T22:48:00.381 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T22:48:00.394 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T22:48:00.396 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T22:48:00.431 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T22:48:00.433 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T22:48:00.729 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T22:48:00.735 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T22:48:00.756 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T22:48:00.762 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T22:48:00.795 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T22:48:00.801 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T22:48:01.116 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T22:48:01.121 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T22:48:01.143 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T22:48:01.149 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T22:48:01.182 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T22:48:01.188 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T22:48:01.499 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T22:48:01.502 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T22:48:01.527 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T22:48:01.530 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T22:48:01.569 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T22:48:01.574 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T22:48:01.839 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T22:48:01.866 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T22:48:01.897 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T22:48:01.986 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T22:48:02.013 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T22:48:02.052 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T22:48:02.209 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T22:48:02.236 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T22:48:02.267 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T22:48:02.684 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T22:48:02.699 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:48:02.733 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:02.948 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T22:48:02.964 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:48:03.006 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:03.053 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T22:48:03.080 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T22:48:03.120 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T22:48:03.342 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T22:48:03.370 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T22:48:03.409 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T22:48:03.456 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T22:48:03.483 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T22:48:03.522 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T22:48:03.771 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T22:48:03.786 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package jq. 2024-09-21T22:48:03.798 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T22:48:03.813 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T22:48:03.837 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T22:48:03.877 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T22:48:04.074 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package jq. 2024-09-21T22:48:04.101 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T22:48:04.130 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package socat. 2024-09-21T22:48:04.131 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T22:48:04.146 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T22:48:04.188 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T22:48:04.359 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package socat. 2024-09-21T22:48:04.375 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T22:48:04.417 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T22:48:04.500 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T22:48:04.515 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T22:48:04.558 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T22:48:04.721 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T22:48:04.749 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T22:48:04.787 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T22:48:04.904 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-test. 2024-09-21T22:48:04.920 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:04.962 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:05.159 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-test. 2024-09-21T22:48:05.187 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:05.225 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:07.267 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T22:48:07.294 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:48:07.333 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:07.489 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T22:48:07.516 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T22:48:07.555 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:07.637 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T22:48:07.665 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:07.804 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T22:48:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:08.115 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T22:48:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:08.142 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T22:48:08.181 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T22:48:08.478 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T22:48:08.479 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T22:48:08.493 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T22:48:08.507 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T22:48:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T22:48:08.554 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T22:48:08.890 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T22:48:08.917 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T22:48:08.950 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T22:48:08.956 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T22:48:08.978 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T22:48:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T22:48:09.320 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package pkg-config. 2024-09-21T22:48:09.347 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T22:48:09.378 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T22:48:09.405 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T22:48:09.432 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T22:48:09.471 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T22:48:09.609 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T22:48:09.624 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T22:48:09.664 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T22:48:09.902 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pkg-config. 2024-09-21T22:48:09.929 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T22:48:09.968 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T22:48:10.076 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T22:48:10.103 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T22:48:10.142 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T22:48:10.221 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T22:48:10.249 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T22:48:10.288 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T22:48:10.454 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T22:48:10.481 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T22:48:10.545 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T22:48:10.724 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T22:48:10.752 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T22:48:10.790 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T22:48:10.857 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T22:48:10.884 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T22:48:10.923 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T22:48:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T22:48:11.084 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T22:48:11.118 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T22:48:11.193 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T22:48:11.220 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T22:48:11.260 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T22:48:11.414 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T22:48:11.442 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T22:48:11.481 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T22:48:11.579 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-py. 2024-09-21T22:48:11.607 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T22:48:11.646 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T22:48:11.792 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T22:48:11.820 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T22:48:11.858 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T22:48:11.932 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T22:48:11.960 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T22:48:11.999 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T22:48:12.195 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-py. 2024-09-21T22:48:12.223 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T22:48:12.261 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T22:48:12.582 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T22:48:12.609 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T22:48:12.648 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T22:48:12.670 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T22:48:12.697 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T22:48:12.736 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T22:48:13.047 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-toml. 2024-09-21T22:48:13.075 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T22:48:13.114 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T22:48:13.236 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T22:48:13.263 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T22:48:13.392 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T22:48:13.419 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T22:48:13.427 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T22:48:13.458 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T22:48:13.739 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-toml. 2024-09-21T22:48:13.766 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T22:48:13.805 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T22:48:13.829 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T22:48:13.856 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T22:48:13.903 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T22:48:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T22:48:14.136 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T22:48:14.147 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T22:48:14.164 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T22:48:14.175 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T22:48:14.197 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T22:48:14.570 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T22:48:14.598 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T22:48:14.637 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T22:48:14.889 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T22:48:14.905 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T22:48:14.927 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package radosgw. 2024-09-21T22:48:14.948 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T22:48:14.955 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:14.993 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:15.727 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package radosgw. 2024-09-21T22:48:15.755 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:15.793 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:15.805 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T22:48:15.821 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:15.863 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:16.156 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T22:48:16.393 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T22:48:16.519 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T22:48:16.614 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T22:48:16.642 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T22:48:16.680 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:16.782 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T22:48:16.969 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T22:48:17.141 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T22:48:17.218 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T22:48:17.353 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T22:48:17.600 INFO:teuthology.orchestra.run.smithi099.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T22:48:17.613 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T22:48:17.687 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:17.921 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T22:48:18.179 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user cephadm....done 2024-09-21T22:48:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T22:48:18.323 INFO:teuthology.orchestra.run.smithi094.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T22:48:18.406 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:18.587 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T22:48:18.838 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T22:48:18.956 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T22:48:19.198 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T22:48:19.323 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user cephadm....done 2024-09-21T22:48:19.433 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T22:48:19.480 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T22:48:19.559 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T22:48:19.748 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T22:48:19.870 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T22:48:19.982 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T22:48:20.092 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T22:48:20.255 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T22:48:20.343 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T22:48:20.523 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T22:48:20.611 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T22:48:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T22:48:20.774 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T22:48:21.017 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T22:48:21.064 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T22:48:21.268 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T22:48:21.408 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T22:48:21.528 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T22:48:21.726 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T22:48:21.787 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:22.002 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T22:48:22.122 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T22:48:22.239 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T22:48:22.262 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T22:48:22.340 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T22:48:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T22:48:22.521 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T22:48:22.693 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T22:48:22.764 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T22:48:23.049 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:23.090 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T22:48:23.124 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T22:48:23.188 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T22:48:23.291 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T22:48:23.313 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T22:48:23.426 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T22:48:23.552 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T22:48:23.656 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T22:48:23.678 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T22:48:23.757 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T22:48:23.937 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T22:48:24.042 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T22:48:24.318 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T22:48:24.334 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T22:48:24.368 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T22:48:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T22:48:24.559 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T22:48:24.628 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T22:48:24.754 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T22:48:24.927 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T22:48:25.052 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T22:48:25.354 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T22:48:25.422 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T22:48:25.622 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T22:48:25.681 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T22:48:25.782 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T22:48:25.898 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T22:48:25.917 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T22:48:25.999 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T22:48:26.210 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T22:48:26.586 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T22:48:26.668 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T22:48:26.845 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T22:48:26.927 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T22:48:27.062 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T22:48:27.188 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T22:48:27.222 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T22:48:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T22:48:27.490 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T22:48:27.816 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T22:48:27.832 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T22:48:28.068 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T22:48:28.091 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T22:48:28.210 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:28.477 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T22:48:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T22:48:28.794 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T22:48:28.829 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T22:48:29.163 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T22:48:29.414 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T22:48:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:29.808 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T22:48:30.044 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T22:48:30.126 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T22:48:30.287 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:30.421 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T22:48:30.706 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T22:48:30.807 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T22:48:30.933 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T22:48:31.243 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T22:48:31.290 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T22:48:31.502 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T22:48:31.525 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:31.660 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T22:48:31.745 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:31.871 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T22:48:31.919 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T22:48:32.054 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T22:48:32.172 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T22:48:32.180 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T22:48:32.407 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T22:48:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T22:48:32.658 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T22:48:32.715 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T22:48:32.917 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T22:48:32.950 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:33.076 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T22:48:33.343 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T22:48:33.377 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T22:48:33.611 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T22:48:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T22:48:33.737 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T22:48:33.865 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T22:48:34.047 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T22:48:34.099 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T22:48:34.148 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T22:48:34.359 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T22:48:34.415 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T22:48:34.626 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T22:48:34.725 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T22:48:34.752 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T22:48:34.993 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T22:48:35.028 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T22:48:35.119 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T22:48:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T22:48:35.245 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T22:48:35.335 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T22:48:35.464 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T22:48:35.605 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T22:48:35.706 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T22:48:35.757 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T22:48:35.999 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T22:48:36.082 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T22:48:36.125 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T22:48:36.251 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T22:48:36.340 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T22:48:36.350 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T22:48:36.476 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T22:48:36.586 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T22:48:36.712 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T22:48:36.736 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:36.861 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T22:48:37.114 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T22:48:37.179 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T22:48:37.373 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T22:48:37.464 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T22:48:37.474 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T22:48:37.723 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T22:48:37.733 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T22:48:37.849 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T22:48:37.859 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T22:48:38.126 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T22:48:38.194 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T22:48:38.419 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T22:48:38.511 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T22:48:38.780 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T22:48:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T22:48:38.913 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T22:48:39.140 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T22:48:39.181 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T22:48:39.441 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T22:48:39.442 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T22:48:39.786 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T22:48:39.810 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T22:48:39.912 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:40.038 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:40.139 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T22:48:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T22:48:40.471 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T22:48:40.509 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:40.635 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:40.761 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T22:48:40.824 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T22:48:40.942 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.054 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.068 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.172 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.193 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T22:48:41.281 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.495 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T22:48:41.495 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T22:48:41.530 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.672 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:41.790 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T22:48:41.976 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.050 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.135 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.278 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:42.403 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T22:48:42.514 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T22:48:42.514 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T22:48:42.978 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.137 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.263 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.397 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T22:48:43.611 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.729 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.854 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:43.955 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.073 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.563 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.625 INFO:teuthology.orchestra.run.smithi099.stdout:Adding group ceph....done 2024-09-21T22:48:44.685 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.811 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.937 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:44.993 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user ceph....done 2024-09-21T22:48:45.063 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:45.170 INFO:teuthology.orchestra.run.smithi099.stdout:Setting system user ceph properties....done 2024-09-21T22:48:45.181 INFO:teuthology.orchestra.run.smithi099.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T22:48:45.314 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T22:48:45.564 INFO:teuthology.orchestra.run.smithi094.stdout:Adding group ceph....done 2024-09-21T22:48:45.623 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T22:48:45.899 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user ceph....done 2024-09-21T22:48:46.017 INFO:teuthology.orchestra.run.smithi094.stdout:Setting system user ceph properties....done 2024-09-21T22:48:46.032 INFO:teuthology.orchestra.run.smithi094.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T22:48:46.108 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T22:48:46.234 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T22:48:46.717 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T22:48:46.718 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T22:48:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:47.090 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:47.153 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:47.485 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T22:48:47.538 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T22:48:47.539 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T22:48:47.983 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:47.988 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T22:48:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T22:48:48.335 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T22:48:48.785 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:48.917 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:49.016 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T22:48:49.016 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T22:48:49.129 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T22:48:49.129 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T22:48:49.486 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:49.568 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:49.701 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T22:48:49.701 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T22:48:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T22:48:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T22:48:50.146 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:50.320 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T22:48:50.449 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T22:48:50.471 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:50.613 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:50.827 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T22:48:50.827 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T22:48:50.964 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.074 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.208 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.348 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.416 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T22:48:51.416 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T22:48:51.484 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.610 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.845 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:51.909 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:52.060 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:52.186 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:52.197 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T22:48:52.306 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T22:48:52.337 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T22:48:52.459 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T22:48:52.698 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T22:48:52.849 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T22:48:53.135 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T22:48:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T22:48:53.819 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T22:48:53.824 DEBUG:teuthology.parallel:result is None 2024-09-21T22:48:54.151 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T22:48:54.818 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T22:48:54.823 DEBUG:teuthology.parallel:result is None 2024-09-21T22:48:54.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:48:54.969 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-query -W -f '${Version}' ceph 2024-09-21T22:48:54.987 INFO:teuthology.orchestra.run.smithi094.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:48:54.988 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:48:54.988 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T22:48:54.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:48:55.129 DEBUG:teuthology.orchestra.run.smithi099:> dpkg-query -W -f '${Version}' ceph 2024-09-21T22:48:55.148 INFO:teuthology.orchestra.run.smithi099.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:48:55.148 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:48:55.149 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T22:48:55.150 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T22:48:55.150 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:55.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T22:48:55.164 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:55.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T22:48:55.202 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T22:48:55.203 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:55.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T22:48:55.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T22:48:55.272 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:55.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T22:48:55.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T22:48:55.341 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T22:48:55.341 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:55.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T22:48:55.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T22:48:55.408 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:55.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T22:48:55.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T22:48:55.477 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T22:48:55.477 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:55.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T22:48:55.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T22:48:55.544 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:55.544 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T22:48:55.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T22:48:55.614 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T22:48:55.688 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T22:48:55.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-09-21T22:48:55.691 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-21T22:48:55.703 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T22:48:55.703 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T22:48:55.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T22:48:55.758 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T22:48:55.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T22:48:55.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T22:48:55.825 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T22:48:55.825 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T22:48:55.825 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914492', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOIc4XLKRKA3fh/vlr5hG/eRj+ijFVxgTmfsbyDqzhLo4H9nzQXuFpKibdCq6U05DAN+1rjzG6xkxQtzBpGH4fE=', 'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGT/EY8LA7640P1YsNZ+wmrCB717w/+KCum9Gj/SGN5X3Pfb407TDRJ7NvcSxKK4MKyWqNHcLvu3OI/tI8vB1KQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T22:48:55.825 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T22:48:55.859 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T22:48:55.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T22:48:55.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T22:48:55.920 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:55.920 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T22:48:55.966 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T22:48:55.966 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-21 22:43:28.886474512 +0000 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-21 22:43:28.882474548 +0000 2024-09-21T22:48:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-21 22:43:28.882474548 +0000 2024-09-21T22:48:56.015 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-09-21T22:48:56.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T22:48:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-21T22:48:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-21T22:48:56.067 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000205861 s, 2.5 MB/s 2024-09-21T22:48:56.068 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T22:48:56.115 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-21T22:48:56.165 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T22:48:56.165 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-21 22:43:29.598468153 +0000 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-21 22:43:29.338470475 +0000 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-21 22:43:29.338470475 +0000 2024-09-21T22:48:56.166 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-09-21T22:48:56.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T22:48:56.219 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-21T22:48:56.219 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-21T22:48:56.219 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000315134 s, 1.6 MB/s 2024-09-21T22:48:56.221 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T22:48:56.271 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-21T22:48:56.317 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 919 Links: 1 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-21 22:43:30.062464007 +0000 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-21 22:43:29.818466188 +0000 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-21 22:43:29.818466188 +0000 2024-09-21T22:48:56.318 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-09-21T22:48:56.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T22:48:56.372 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-21T22:48:56.372 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-21T22:48:56.372 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00032393 s, 1.6 MB/s 2024-09-21T22:48:56.373 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T22:48:56.423 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-21 22:43:30.518459934 +0000 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-21 22:43:30.278462079 +0000 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-21 22:43:30.278462079 +0000 2024-09-21T22:48:56.470 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-09-21T22:48:56.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T22:48:56.524 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-21T22:48:56.524 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-21T22:48:56.524 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00026073 s, 2.0 MB/s 2024-09-21T22:48:56.525 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T22:48:56.575 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:56.576 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:56.576 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T22:48:56.582 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T22:48:56.582 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 22:43:32.725806839 +0000 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 22:43:32.721806874 +0000 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 22:43:32.721806874 +0000 2024-09-21T22:48:56.631 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-21T22:48:56.632 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T22:48:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T22:48:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T22:48:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000340456 s, 1.5 MB/s 2024-09-21T22:48:56.686 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T22:48:56.735 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-21T22:48:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T22:48:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.782 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-21T22:48:56.783 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.783 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 22:43:33.505799872 +0000 2024-09-21T22:48:56.783 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 22:43:33.209802516 +0000 2024-09-21T22:48:56.783 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 22:43:33.209802516 +0000 2024-09-21T22:48:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-21T22:48:56.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T22:48:56.837 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T22:48:56.837 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T22:48:56.837 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000403849 s, 1.3 MB/s 2024-09-21T22:48:56.838 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T22:48:56.887 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 22:43:33.989795548 +0000 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 22:43:33.733797835 +0000 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 22:43:33.733797835 +0000 2024-09-21T22:48:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-21T22:48:56.934 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T22:48:56.988 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T22:48:56.988 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T22:48:56.988 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.00038975 s, 1.3 MB/s 2024-09-21T22:48:56.989 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T22:48:57.039 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 22:43:34.461791331 +0000 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 22:43:34.233793369 +0000 2024-09-21T22:48:57.086 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 22:43:34.233793369 +0000 2024-09-21T22:48:57.087 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-21T22:48:57.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T22:48:57.140 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T22:48:57.140 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T22:48:57.141 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000319114 s, 1.6 MB/s 2024-09-21T22:48:57.141 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T22:48:57.192 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:57.192 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T22:48:57.192 INFO:tasks.ceph:Generating config... 2024-09-21T22:48:57.193 INFO:tasks.ceph:[client] rbd cache = False 2024-09-21T22:48:57.193 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T22:48:57.193 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T22:48:57.193 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T22:48:57.193 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T22:48:57.193 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T22:48:57.193 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T22:48:57.193 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T22:48:57.193 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T22:48:57.193 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T22:48:57.193 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T22:48:57.194 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T22:48:57.195 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T22:48:57.195 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T22:48:57.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T22:48:57.228 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T22:48:57.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T22:48:57.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T22:48:57.372 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.c', '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]'), ('mon.b', '172.21.15.99')] 2024-09-21T22:48:57.372 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.99', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T22:48:57.373 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:57.373 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T22:48:57.417 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --addv c '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]' --add b 172.21.15.99 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:48:57.501 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:48:57.501 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid 56013142-0739-4aee-ba29-30eddcc2dbe0 2024-09-21T22:48:57.501 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-09-21T22:48:57.501 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 56013142-0739-4aee-ba29-30eddcc2dbe0 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-09-21T22:48:57.500447+0000 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-21T22:48:57.500447+0000 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] mon.c 2024-09-21T22:48:57.502 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T22:48:57.503 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T22:48:57.550 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 56013142-0739-4aee-ba29-30eddcc2dbe0... 2024-09-21T22:48:57.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T22:48:57.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-21T22:48:57.623 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.99" 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon client directed command retry = 5 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on pool no app = False 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: ms inject socket failures = 5000 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 56013142-0739-4aee-ba29-30eddcc2dbe0 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-21T22:48:57.626 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-21T22:48:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = stupid 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-21T22:48:57.628 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: osd blocked scrub grace period = 3600 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: osd max markdown count = 1000 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: osd shutdown pgref assert = True 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-21T22:48:57.629 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-21T22:48:57.630 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:48:57.631 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout: rbd cache = False 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-21T22:48:57.632 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-09-21T22:48:57.633 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-09-21T22:48:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-09-21T22:48:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.99" 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: mon client directed command retry = 5 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on pool no app = False 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: ms inject socket failures = 5000 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 56013142-0739-4aee-ba29-30eddcc2dbe0 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug trim objects = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = stupid 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2024-09-21T22:48:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: osd blocked scrub grace period = 3600 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: osd max markdown count = 1000 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: osd shutdown pgref assert = True 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-21T22:48:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T22:48:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T22:48:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T22:48:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: rbd cache = False 2024-09-21T22:48:57.642 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-09-21T22:48:57.642 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-09-21T22:48:57.642 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-09-21T22:48:57.643 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T22:48:57.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T22:48:57.733 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T22:48:57.733 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:57.733 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T22:48:57.782 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:57.782 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T22:48:57.829 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:48:57.829 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:57.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T22:48:57.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T22:48:57.894 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:48:57.894 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:48:57.941 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:48:57.942 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:57.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T22:48:57.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T22:48:57.965 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:48:57.965 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:48:58.014 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T22:48:58.014 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T22:48:58.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T22:48:58.058 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T22:48:58.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T22:48:58.103 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T22:48:58.117 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T22:48:58.118 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T22:48:58.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T22:48:58.154 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T22:48:58.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T22:48:58.199 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T22:48:58.209 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T22:48:58.210 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T22:48:58.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T22:48:58.222 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:58.222 INFO:tasks.ceph:role: osd.0 2024-09-21T22:48:58.222 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:48:58.222 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:48:58.276 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:48:58.277 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:48:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:48:58.277 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:48:58.306 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-21T22:48:58.308 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-21T22:48:58.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T22:48:58.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T22:48:58.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:48:58.416 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:48:58.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T22:48:58.467 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:58.467 INFO:tasks.ceph:role: osd.1 2024-09-21T22:48:58.467 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:48:58.468 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T22:48:58.521 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:48:58.522 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:48:58.550 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-21T22:48:58.551 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-21T22:48:58.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T22:48:58.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T22:48:58.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:48:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:48:58.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T22:48:58.721 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:58.721 INFO:tasks.ceph:role: osd.2 2024-09-21T22:48:58.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:48:58.721 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:48:58.775 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:48:58.805 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-21T22:48:58.806 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-21T22:48:58.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T22:48:58.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T22:48:58.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:48:58.918 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:48:58.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T22:48:58.971 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T22:48:58.971 INFO:tasks.ceph:role: osd.3 2024-09-21T22:48:58.971 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:48:58.971 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T22:48:59.025 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:48:59.026 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:48:59.054 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-21T22:48:59.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-21T22:48:59.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T22:48:59.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T22:48:59.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:48:59.166 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:48:59.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:48:59.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:48:59.277+0000 7fc52c1787c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T22:48:59.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:48:59.277+0000 7fc52c1787c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T22:48:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:48:59.277+0000 7fc52c1787c0 -1 bdev(0x55bbeb3cd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T22:48:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:48:59.277+0000 7fc52c1787c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T22:49:02.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T22:49:02.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:02.793+0000 7f7c1bfea7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T22:49:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:02.793+0000 7f7c1bfea7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T22:49:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:02.793+0000 7f7c1bfea7c0 -1 bdev(0x563abd555800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:02.793+0000 7f7c1bfea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T22:49:05.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T22:49:05.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:06.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:06.065+0000 7f99f10ba7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T22:49:06.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:06.065+0000 7f99f10ba7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T22:49:06.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:06.065+0000 7f99f10ba7c0 -1 bdev(0x5642c52ad800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:06.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:06.065+0000 7f99f10ba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T22:49:09.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T22:49:09.293 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:09.373+0000 7f5c2fea27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T22:49:09.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:09.373+0000 7f5c2fea27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T22:49:09.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:09.373+0000 7f5c2fea27c0 -1 bdev(0x557ee56e3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:09.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:49:09.373+0000 7f5c2fea27c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T22:49:12.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T22:49:12.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T22:49:12.591 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T22:49:12.591 INFO:tasks.ceph:role: osd.4 2024-09-21T22:49:12.591 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:49:12.591 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:49:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:49:13.145 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-21T22:49:13.147 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-21T22:49:13.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T22:49:13.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T22:49:13.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:49:13.319 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:49:13.319 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T22:49:13.372 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T22:49:13.372 INFO:tasks.ceph:role: osd.5 2024-09-21T22:49:13.372 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:49:13.373 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T22:49:13.433 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:49:13.433 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:49:13.433 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:49:13.434 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:49:13.879 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-21T22:49:13.881 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-21T22:49:13.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T22:49:13.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T22:49:14.016 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:49:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:49:14.017 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T22:49:14.069 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T22:49:14.069 INFO:tasks.ceph:role: osd.6 2024-09-21T22:49:14.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:49:14.069 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:49:14.134 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:49:14.596 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-21T22:49:14.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-21T22:49:14.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T22:49:14.720 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T22:49:14.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:49:14.732 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:49:14.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T22:49:14.785 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T22:49:14.785 INFO:tasks.ceph:role: osd.7 2024-09-21T22:49:14.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:49:14.785 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T22:49:14.850 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T22:49:14.850 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T22:49:14.850 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T22:49:14.850 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T22:49:14.850 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T22:49:14.851 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-21T22:49:14.851 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T22:49:14.851 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T22:49:14.851 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T22:49:14.851 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T22:49:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-21T22:49:15.344 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-21T22:49:15.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T22:49:15.486 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T22:49:15.497 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T22:49:15.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:49:15.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:15.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:15.602+0000 7f9d02cc57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T22:49:15.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:15.606+0000 7f9d02cc57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T22:49:15.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:15.606+0000 7f9d02cc57c0 -1 bdev(0x55ff6a765800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:15.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:15.606+0000 7f9d02cc57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T22:49:18.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T22:49:18.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:18.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:18.858+0000 7f6178fdf7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T22:49:18.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:18.858+0000 7f6178fdf7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T22:49:18.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:18.858+0000 7f6178fdf7c0 -1 bdev(0x55f3d2b51800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:18.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:18.858+0000 7f6178fdf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T22:49:22.045 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T22:49:22.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:22.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:22.178+0000 7f6d1ad4c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T22:49:22.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:22.178+0000 7f6d1ad4c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T22:49:22.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:22.178+0000 7f6d1ad4c7c0 -1 bdev(0x560b9c9f3800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:22.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:22.178+0000 7f6d1ad4c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T22:49:25.330 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T22:49:25.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:25.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:25.462+0000 7fa7760f17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T22:49:25.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:25.462+0000 7fa7760f17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T22:49:25.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:25.462+0000 7fa7760f17c0 -1 bdev(0x55b4f4a4d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T22:49:25.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T22:49:25.462+0000 7fa7760f17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T22:49:28.877 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T22:49:28.934 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T22:49:28.934 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:28.934 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T22:49:28.949 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:28.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T22:49:29.004 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:29.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T22:49:29.056 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:29.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T22:49:29.112 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:29.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T22:49:29.164 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T22:49:29.177 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T22:49:29.233 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T22:49:29.285 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T22:49:29.341 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T22:49:29.396 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:29.396 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T22:49:29.402 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:29.402 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T22:49:29.441 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T22:49:29.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T22:49:29.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T22:49:29.487 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-09-21T22:49:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBaTe9mw+5lAxAAo46cm6sA4TXZFtTHA6hd9Q== 2024-09-21T22:49:29.487 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBbTe9mVhvnEBAALKwfhDQTZrWnB6f2kCyyig== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBeTe9mN1+aLxAAVdh/m2YlguoWkf8fOTQyhw== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBiTe9m/KA5BBAA3Q3R90xHJKNDvG1Pc7kjsQ== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBlTe9masKBFhAAeW1jcGmL7jwINkLe9q/VTA== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBaTe9mOL8rBhAA7hKAZLPGZUtuv9WfXOynKg== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBrTe9mRrVOJBAAt1miG+TI/oCBUjM0G3Nhgw== 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-09-21T22:49:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBuTe9mZGNzMxAAGL0Uxxtt7a1qV67lWwdnew== 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQByTe9mFenbChAAGevtXPQg7Ioo6A5nbgf9Xw== 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQB1Te9masbWGxAAMJ2rZCidZUogefzD7++ecQ== 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBaTe9mb00eCRAAK5D1Om/0uHfFjpHb6B6FVQ== 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-09-21T22:49:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBaTe9mHQXlCxAAEiGbPlRxbnjngrjV2rB17A== 2024-09-21T22:49:29.495 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-09-21T22:49:29.495 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBaTe9mw+5lAxAAo46cm6sA4TXZFtTHA6hd9Q== 2024-09-21T22:49:29.495 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-09-21T22:49:29.495 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBbTe9mVhvnEBAALKwfhDQTZrWnB6f2kCyyig== 2024-09-21T22:49:29.495 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBeTe9mN1+aLxAAVdh/m2YlguoWkf8fOTQyhw== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBiTe9m/KA5BBAA3Q3R90xHJKNDvG1Pc7kjsQ== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBlTe9masKBFhAAeW1jcGmL7jwINkLe9q/VTA== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBaTe9mOL8rBhAA7hKAZLPGZUtuv9WfXOynKg== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBrTe9mRrVOJBAAt1miG+TI/oCBUjM0G3Nhgw== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBuTe9mZGNzMxAAGL0Uxxtt7a1qV67lWwdnew== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQByTe9mFenbChAAGevtXPQg7Ioo6A5nbgf9Xw== 2024-09-21T22:49:29.496 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-09-21T22:49:29.497 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB1Te9masbWGxAAMJ2rZCidZUogefzD7++ecQ== 2024-09-21T22:49:29.497 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-21T22:49:29.497 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBaTe9mb00eCRAAK5D1Om/0uHfFjpHb6B6FVQ== 2024-09-21T22:49:29.497 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-21T22:49:29.497 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBaTe9mHQXlCxAAEiGbPlRxbnjngrjV2rB17A== 2024-09-21T22:49:29.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T22:49:29.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T22:49:29.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.587 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:29.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T22:49:29.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T22:49:30.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T22:49:30.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T22:49:30.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T22:49:30.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T22:49:30.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T22:49:30.478 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T22:49:30.478 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T22:49:30.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T22:49:30.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T22:49:30.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T22:49:30.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T22:49:30.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T22:49:30.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T22:49:30.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T22:49:30.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T22:49:30.871 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:30.873 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T22:49:30.921 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T22:49:30.922 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T22:49:30.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T22:49:30.924 INFO:tasks.ceph.mon.a:Started 2024-09-21T22:49:30.924 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T22:49:30.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T22:49:30.926 INFO:tasks.ceph.mon.c:Started 2024-09-21T22:49:30.927 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T22:49:30.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T22:49:30.970 INFO:tasks.ceph.mon.b:Started 2024-09-21T22:49:30.970 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T22:49:30.970 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T22:49:30.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T22:49:30.972 INFO:tasks.ceph.mgr.y:Started 2024-09-21T22:49:30.973 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T22:49:30.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T22:49:30.975 INFO:tasks.ceph.mgr.x:Started 2024-09-21T22:49:30.975 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T22:49:30.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T22:49:36.796 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:36.790+0000 7f8100c3f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T22:49:37.183 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-09-21T22:49:37.203 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T22:49:37.203 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T22:49:37.203 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:37.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T22:49:37.214 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:37.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T22:49:37.272 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:37.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T22:49:37.308 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:37.301+0000 7fcc45701180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T22:49:37.319 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-21T22:49:37.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T22:49:37.352 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:37.346+0000 7f8100c3f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T22:49:37.372 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:37.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T22:49:37.385 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:37.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T22:49:37.442 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:37.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T22:49:37.496 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T22:49:37.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T22:49:37.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new ecfcea04-7198-4c57-bc74-45aaf791e3b7 0 2024-09-21T22:49:37.568 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:37.562+0000 7f8100c3f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T22:49:37.785 INFO:teuthology.orchestra.run.smithi099.stdout:0 2024-09-21T22:49:37.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new b6b749e6-4a66-4d8b-bf50-c9634a6927d5 1 2024-09-21T22:49:37.841 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:37.837+0000 7fcc45701180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T22:49:37.999 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-21T22:49:38.017 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 4e90da65-2fdd-4c85-a207-70d5f22c95a4 2 2024-09-21T22:49:38.059 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.053+0000 7fcc45701180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T22:49:38.154 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.150+0000 7f8100c3f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T22:49:38.207 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.202+0000 7f8100c3f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T22:49:38.218 INFO:teuthology.orchestra.run.smithi099.stdout:2 2024-09-21T22:49:38.235 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 3c97651a-402c-4171-9648-c615dd754c94 3 2024-09-21T22:49:38.263 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.258+0000 7f8100c3f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T22:49:38.343 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.338+0000 7f8100c3f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T22:49:38.396 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.390+0000 7f8100c3f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T22:49:38.423 INFO:teuthology.orchestra.run.smithi099.stdout:3 2024-09-21T22:49:38.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new b935b00e-0d91-4f2b-9f2d-2c2be1144ca8 4 2024-09-21T22:49:38.475 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.470+0000 7f8100c3f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T22:49:38.530 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.526+0000 7f8100c3f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T22:49:38.632 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.625+0000 7fcc45701180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T22:49:38.677 INFO:teuthology.orchestra.run.smithi099.stdout:4 2024-09-21T22:49:38.686 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.682+0000 7f8100c3f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T22:49:38.686 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.681+0000 7fcc45701180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T22:49:38.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 559dea61-5abd-423e-9dfb-cf92f9e6d4a4 5 2024-09-21T22:49:38.739 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.733+0000 7fcc45701180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T22:49:38.749 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:38.746+0000 7f8100c3f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T22:49:38.816 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.809+0000 7fcc45701180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T22:49:38.868 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.861+0000 7fcc45701180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T22:49:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:5 2024-09-21T22:49:38.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new aeee922d-53c2-41f4-9433-8a9691f3e674 6 2024-09-21T22:49:38.947 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.941+0000 7fcc45701180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T22:49:39.000 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:38.993+0000 7fcc45701180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T22:49:39.106 INFO:teuthology.orchestra.run.smithi099.stdout:6 2024-09-21T22:49:39.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new b40a3234-6950-4a73-9a5c-f0c086450816 7 2024-09-21T22:49:39.152 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:39.145+0000 7fcc45701180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T22:49:39.213 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:39.209+0000 7fcc45701180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T22:49:39.323 INFO:teuthology.orchestra.run.smithi099.stdout:7 2024-09-21T22:49:39.339 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T22:49:39.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T22:49:39.341 INFO:tasks.ceph.osd.0:Started 2024-09-21T22:49:39.342 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T22:49:39.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T22:49:39.344 INFO:tasks.ceph.osd.1:Started 2024-09-21T22:49:39.344 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T22:49:39.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T22:49:39.346 INFO:tasks.ceph.osd.2:Started 2024-09-21T22:49:39.346 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T22:49:39.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T22:49:39.349 INFO:tasks.ceph.osd.3:Started 2024-09-21T22:49:39.349 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T22:49:39.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T22:49:39.351 INFO:tasks.ceph.osd.4:Started 2024-09-21T22:49:39.351 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T22:49:39.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T22:49:39.353 INFO:tasks.ceph.osd.5:Started 2024-09-21T22:49:39.353 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T22:49:39.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T22:49:39.355 INFO:tasks.ceph.osd.6:Started 2024-09-21T22:49:39.355 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T22:49:39.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T22:49:39.357 INFO:tasks.ceph.osd.7:Started 2024-09-21T22:49:39.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:39.419 INFO:tasks.ceph.mgr.x.smithi099.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T22:49:39.419 INFO:tasks.ceph.mgr.x.smithi099.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T22:49:39.420 INFO:tasks.ceph.mgr.x.smithi099.stderr: from numpy import show_config as show_numpy_config 2024-09-21T22:49:39.465 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:39.462+0000 7f8100c3f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T22:49:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:49:39.582 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:39.319546+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:39.601 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:39.594+0000 7f8100c3f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T22:49:39.605 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T22:49:39.605 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T22:49:39.764 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:39.758+0000 7f8100c3f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T22:49:39.827 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:39.822+0000 7f8100c3f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T22:49:39.889 INFO:tasks.ceph.mgr.y.smithi094.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T22:49:39.889 INFO:tasks.ceph.mgr.y.smithi094.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T22:49:39.889 INFO:tasks.ceph.mgr.y.smithi094.stderr: from numpy import show_config as show_numpy_config 2024-09-21T22:49:39.957 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:39.953+0000 7fcc45701180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T22:49:39.960 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:39.954+0000 7f8100c3f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T22:49:40.132 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:40.125+0000 7fcc45701180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T22:49:40.293 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:40.289+0000 7fcc45701180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T22:49:40.358 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:40.353+0000 7fcc45701180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T22:49:40.410 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:40.497 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:40.493+0000 7fcc45701180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T22:49:40.556 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:40.550+0000 7f8100c3f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T22:49:40.584 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-21T22:49:40.584 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:39.319546+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:40.599 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T22:49:40.631 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:40.626+0000 7f8100c3f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T22:49:40.812 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:40.806+0000 7f8100c3f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T22:49:40.884 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:40.878+0000 7f8100c3f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T22:49:40.994 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:49:40.990+0000 7f28b2dbc7c0 -1 Falling back to public interface 2024-09-21T22:49:41.038 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:49:41.033+0000 7f8b596697c0 -1 Falling back to public interface 2024-09-21T22:49:41.107 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.101+0000 7fcc45701180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T22:49:41.176 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:41.170+0000 7f8100c3f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T22:49:41.182 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.177+0000 7fcc45701180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T22:49:41.237 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:41.234+0000 7f8100c3f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T22:49:41.238 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:49:41.233+0000 7fc4381137c0 -1 Falling back to public interface 2024-09-21T22:49:41.253 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:49:41.249+0000 7f30c1fa27c0 -1 Falling back to public interface 2024-09-21T22:49:41.314 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:49:41.309+0000 7f335aaa07c0 -1 Falling back to public interface 2024-09-21T22:49:41.376 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.369+0000 7fcc45701180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T22:49:41.450 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.445+0000 7fcc45701180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T22:49:41.499 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:49:41.494+0000 7f91dd9a67c0 -1 Falling back to public interface 2024-09-21T22:49:41.522 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:49:41.518+0000 7fc6588257c0 -1 Falling back to public interface 2024-09-21T22:49:41.522 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:49:41.518+0000 7fc7f49877c0 -1 Falling back to public interface 2024-09-21T22:49:41.744 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.737+0000 7fcc45701180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T22:49:41.806 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-21T22:49:41.801+0000 7fcc45701180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T22:49:42.285 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:42.282+0000 7f80a7878640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T22:49:42.286 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-21T22:49:42.282+0000 7f80a7878640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T22:49:45.380 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:49:45.373+0000 7f8b596697c0 -1 osd.3 0 log_to_monitors true 2024-09-21T22:49:45.778 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:49:45.774+0000 7f28b2dbc7c0 -1 osd.7 0 log_to_monitors true 2024-09-21T22:49:45.798 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:49:45.794+0000 7fc7f49877c0 -1 osd.5 0 log_to_monitors true 2024-09-21T22:49:45.809 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:49:45.805+0000 7fc4381137c0 -1 osd.0 0 log_to_monitors true 2024-09-21T22:49:45.833 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:49:45.829+0000 7f30c1fa27c0 -1 osd.2 0 log_to_monitors true 2024-09-21T22:49:45.889 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:49:45.886+0000 7f91dd9a67c0 -1 osd.6 0 log_to_monitors true 2024-09-21T22:49:45.977 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:49:45.974+0000 7fc6588257c0 -1 osd.4 0 log_to_monitors true 2024-09-21T22:49:46.133 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:49:46.129+0000 7f335aaa07c0 -1 osd.1 0 log_to_monitors true 2024-09-21T22:49:47.378 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:49:47.373+0000 7f8b55e24640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T22:49:47.405 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:47.406 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:49:47.401+0000 7f8b4d436640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:47.749 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-21T22:49:47.749 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":13,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:47.437028+0000","last_up_change":"2024-09-21T22:49:47.437028+0000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6801","nonce":2344760438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6803","nonce":2344760438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6807","nonce":2344760438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6805","nonce":2344760438}]},"public_addr":"172.21.15.94:6801/2344760438","cluster_addr":"172.21.15.94:6803/2344760438","heartbeat_back_addr":"172.21.15.94:6807/2344760438","heartbeat_front_addr":"172.21.15.94:6805/2344760438","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:47.770 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T22:49:48.879 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:49:48.873+0000 7f30bd728640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T22:49:48.892 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:49:48.885+0000 7fc4348ce640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T22:49:48.894 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:49:48.889+0000 7f335725b640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T22:49:48.909 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:49:48.905+0000 7f30b5d6f640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:48.921 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:49:48.917+0000 7fc42bee0640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:48.921 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:49:48.917+0000 7f334e86d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:49.714 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:49:49.710+0000 7f28ae542640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T22:49:49.747 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:49:49.742+0000 7f28a6b89640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:49.806 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:49:49.802+0000 7f91d912c640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T22:49:49.829 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:49:49.826+0000 7f91d1773640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:49.856 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:49:49.850+0000 7fc653fab640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T22:49:49.882 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:49:49.878+0000 7fc7f010d640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T22:49:49.885 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:49:49.882+0000 7fc64c5f2640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:49.909 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:49:49.906+0000 7fc7e8754640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:49:54.575 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:54.870 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-21T22:49:54.870 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":19,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:53.924937+0000","last_up_change":"2024-09-21T22:49:50.905334+0000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6801","nonce":2344760438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6803","nonce":2344760438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6807","nonce":2344760438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6805","nonce":2344760438}]},"public_addr":"172.21.15.94:6801/2344760438","cluster_addr":"172.21.15.94:6803/2344760438","heartbeat_back_addr":"172.21.15.94:6807/2344760438","heartbeat_front_addr":"172.21.15.94:6805/2344760438","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6811","nonce":545441398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6813","nonce":545441398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6817","nonce":545441398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6815","nonce":545441398}]},"public_addr":"172.21.15.99:6811/545441398","cluster_addr":"172.21.15.99:6813/545441398","heartbeat_back_addr":"172.21.15.99:6817/545441398","heartbeat_front_addr":"172.21.15.99:6815/545441398","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:54.887 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T22:49:54.887 INFO:tasks.ceph:Creating RBD pool 2024-09-21T22:49:54.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T22:49:55.965 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'rbd' created 2024-09-21T22:49:55.990 DEBUG:teuthology.orchestra.run.smithi094:> rbd --cluster ceph pool init rbd 2024-09-21T22:49:59.015 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T22:49:59.016 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T22:49:59.016 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T22:49:59.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T22:49:59.016 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T22:49:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:49:59.386 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3636737213},{"type":"v1","addr":"172.21.15.99:6809","nonce":3636737213}]},"active_addr":"172.21.15.99:6809/3636737213","active_change":"2024-09-21T22:49:41.258784+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2754902787}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3519759960}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1347339918}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1822508428}]}]} 2024-09-21T22:49:59.389 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T22:49:59.389 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T22:49:59.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:59.657 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:49:59.657 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:58.971136+0000","last_up_change":"2024-09-21T22:49:50.905334+0000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T22:49:55.226638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6801","nonce":2344760438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6803","nonce":2344760438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6807","nonce":2344760438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6805","nonce":2344760438}]},"public_addr":"172.21.15.94:6801/2344760438","cluster_addr":"172.21.15.94:6803/2344760438","heartbeat_back_addr":"172.21.15.94:6807/2344760438","heartbeat_front_addr":"172.21.15.94:6805/2344760438","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6811","nonce":545441398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6813","nonce":545441398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6817","nonce":545441398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6815","nonce":545441398}]},"public_addr":"172.21.15.99:6811/545441398","cluster_addr":"172.21.15.99:6813/545441398","heartbeat_back_addr":"172.21.15.99:6817/545441398","heartbeat_front_addr":"172.21.15.99:6815/545441398","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:59.674 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T22:49:59.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:49:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:49:59.954 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:49:58.971136+0000","last_up_change":"2024-09-21T22:49:50.905334+0000","last_in_change":"2024-09-21T22:49:39.319546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T22:49:55.226638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6801","nonce":2344760438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6803","nonce":2344760438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6807","nonce":2344760438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2344760438},{"type":"v1","addr":"172.21.15.94:6805","nonce":2344760438}]},"public_addr":"172.21.15.94:6801/2344760438","cluster_addr":"172.21.15.94:6803/2344760438","heartbeat_back_addr":"172.21.15.94:6807/2344760438","heartbeat_front_addr":"172.21.15.94:6805/2344760438","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6811","nonce":545441398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6813","nonce":545441398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6817","nonce":545441398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":545441398},{"type":"v1","addr":"172.21.15.99:6815","nonce":545441398}]},"public_addr":"172.21.15.99:6811/545441398","cluster_addr":"172.21.15.99:6813/545441398","heartbeat_back_addr":"172.21.15.99:6817/545441398","heartbeat_front_addr":"172.21.15.99:6815/545441398","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:49:59.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T22:49:59.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T22:49:59.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T22:49:59.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T22:49:59.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T22:49:59.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T22:49:59.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T22:49:59.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T22:50:00.231 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T22:50:00.233 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T22:50:00.237 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:00.237 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T22:50:00.239 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:00.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T22:50:00.244 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:00.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T22:50:00.253 INFO:teuthology.orchestra.run.smithi094.stdout:68719476740 2024-09-21T22:50:00.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T22:50:00.256 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:00.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T22:50:00.258 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-09-21T22:50:00.258 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T22:50:00.730 INFO:teuthology.orchestra.run.smithi094.stdout:64424509443 2024-09-21T22:50:00.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2024-09-21T22:50:00.755 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:64424509443 2024-09-21T22:50:00.764 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-21T22:50:00.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2024-09-21T22:50:00.788 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-09-21T22:50:00.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2024-09-21T22:50:00.809 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-21T22:50:00.814 INFO:teuthology.orchestra.run.smithi094.stdout:55834574852 2024-09-21T22:50:00.819 INFO:teuthology.orchestra.run.smithi094.stdout:68719476739 2024-09-21T22:50:00.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.1 2024-09-21T22:50:00.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-21T22:50:00.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.5 2024-09-21T22:50:01.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T22:50:01.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T22:50:01.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T22:50:01.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T22:50:01.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T22:50:01.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T22:50:01.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T22:50:01.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T22:50:02.264 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:02.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.2 2024-09-21T22:50:02.291 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.303 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-21T22:50:02.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.4 2024-09-21T22:50:02.328 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.335 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-21T22:50:02.342 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-21T22:50:02.347 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-21T22:50:02.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.0 2024-09-21T22:50:02.355 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.7 2024-09-21T22:50:02.367 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.6 2024-09-21T22:50:02.372 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-09-21T22:50:02.372 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.373 INFO:teuthology.orchestra.run.smithi094.stdout:68719476740 2024-09-21T22:50:02.388 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-21T22:50:02.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-21T22:50:02.391 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476740 for osd.5 2024-09-21T22:50:02.392 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.1 2024-09-21T22:50:02.405 DEBUG:teuthology.parallel:result is None 2024-09-21T22:50:02.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T22:50:02.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:50:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:02.714 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:50:02.728 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T22:50:01.270384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235880,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3063808,"data_stored":1812110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.311852"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.207004+0000","last_change":"2024-09-21T22:49:59.207004+0000","last_active":"2024-09-21T22:49:59.207004+0000","last_peered":"2024-09-21T22:49:59.207004+0000","last_clean":"2024-09-21T22:49:59.207004+0000","last_became_active":"2024-09-21T22:49:56.978761+0000","last_became_peered":"2024-09-21T22:49:56.978761+0000","last_unstale":"2024-09-21T22:49:59.207004+0000","last_undegraded":"2024-09-21T22:49:59.207004+0000","last_fullsized":"2024-09-21T22:49:59.207004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:02.238042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068420099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995262+0000","last_change":"2024-09-21T22:49:58.995262+0000","last_active":"2024-09-21T22:49:58.995262+0000","last_peered":"2024-09-21T22:49:58.995262+0000","last_clean":"2024-09-21T22:49:58.995262+0000","last_became_active":"2024-09-21T22:49:56.975265+0000","last_became_peered":"2024-09-21T22:49:56.975265+0000","last_unstale":"2024-09-21T22:49:58.995262+0000","last_undegraded":"2024-09-21T22:49:58.995262+0000","last_fullsized":"2024-09-21T22:49:58.995262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:13:34.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986824+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:49:58.986824+0000","last_peered":"2024-09-21T22:49:58.986824+0000","last_clean":"2024-09-21T22:49:58.986824+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:49:58.986824+0000","last_undegraded":"2024-09-21T22:49:58.986824+0000","last_fullsized":"2024-09-21T22:49:58.986824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995224+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:49:58.995224+0000","last_peered":"2024-09-21T22:49:58.995224+0000","last_clean":"2024-09-21T22:49:58.995224+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:49:58.995224+0000","last_undegraded":"2024-09-21T22:49:58.995224+0000","last_fullsized":"2024-09-21T22:49:58.995224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.992797+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:49:58.992797+0000","last_peered":"2024-09-21T22:49:58.992797+0000","last_clean":"2024-09-21T22:49:58.992797+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:49:58.992797+0000","last_undegraded":"2024-09-21T22:49:58.992797+0000","last_fullsized":"2024-09-21T22:49:58.992797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.203552+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:49:59.203552+0000","last_peered":"2024-09-21T22:49:59.203552+0000","last_clean":"2024-09-21T22:49:59.203552+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:49:59.203552+0000","last_undegraded":"2024-09-21T22:49:59.203552+0000","last_fullsized":"2024-09-21T22:49:59.203552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986710+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:49:58.986710+0000","last_peered":"2024-09-21T22:49:58.986710+0000","last_clean":"2024-09-21T22:49:58.986710+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:49:58.986710+0000","last_undegraded":"2024-09-21T22:49:58.986710+0000","last_fullsized":"2024-09-21T22:49:58.986710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.991381+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:49:58.991381+0000","last_peered":"2024-09-21T22:49:58.991381+0000","last_clean":"2024-09-21T22:49:58.991381+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:49:58.991381+0000","last_undegraded":"2024-09-21T22:49:58.991381+0000","last_fullsized":"2024-09-21T22:49:58.991381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.985930+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:49:58.985930+0000","last_peered":"2024-09-21T22:49:58.985930+0000","last_clean":"2024-09-21T22:49:58.985930+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:49:58.985930+0000","last_undegraded":"2024-09-21T22:49:58.985930+0000","last_fullsized":"2024-09-21T22:49:58.985930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476740,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:50:02.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:50:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:03.002 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:50:03.019 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T22:50:01.270384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235880,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3063808,"data_stored":1812110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.311852"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.207004+0000","last_change":"2024-09-21T22:49:59.207004+0000","last_active":"2024-09-21T22:49:59.207004+0000","last_peered":"2024-09-21T22:49:59.207004+0000","last_clean":"2024-09-21T22:49:59.207004+0000","last_became_active":"2024-09-21T22:49:56.978761+0000","last_became_peered":"2024-09-21T22:49:56.978761+0000","last_unstale":"2024-09-21T22:49:59.207004+0000","last_undegraded":"2024-09-21T22:49:59.207004+0000","last_fullsized":"2024-09-21T22:49:59.207004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:02.238042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068420099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995262+0000","last_change":"2024-09-21T22:49:58.995262+0000","last_active":"2024-09-21T22:49:58.995262+0000","last_peered":"2024-09-21T22:49:58.995262+0000","last_clean":"2024-09-21T22:49:58.995262+0000","last_became_active":"2024-09-21T22:49:56.975265+0000","last_became_peered":"2024-09-21T22:49:56.975265+0000","last_unstale":"2024-09-21T22:49:58.995262+0000","last_undegraded":"2024-09-21T22:49:58.995262+0000","last_fullsized":"2024-09-21T22:49:58.995262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:13:34.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986824+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:49:58.986824+0000","last_peered":"2024-09-21T22:49:58.986824+0000","last_clean":"2024-09-21T22:49:58.986824+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:49:58.986824+0000","last_undegraded":"2024-09-21T22:49:58.986824+0000","last_fullsized":"2024-09-21T22:49:58.986824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995224+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:49:58.995224+0000","last_peered":"2024-09-21T22:49:58.995224+0000","last_clean":"2024-09-21T22:49:58.995224+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:49:58.995224+0000","last_undegraded":"2024-09-21T22:49:58.995224+0000","last_fullsized":"2024-09-21T22:49:58.995224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.992797+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:49:58.992797+0000","last_peered":"2024-09-21T22:49:58.992797+0000","last_clean":"2024-09-21T22:49:58.992797+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:49:58.992797+0000","last_undegraded":"2024-09-21T22:49:58.992797+0000","last_fullsized":"2024-09-21T22:49:58.992797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.203552+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:49:59.203552+0000","last_peered":"2024-09-21T22:49:59.203552+0000","last_clean":"2024-09-21T22:49:59.203552+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:49:59.203552+0000","last_undegraded":"2024-09-21T22:49:59.203552+0000","last_fullsized":"2024-09-21T22:49:59.203552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986710+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:49:58.986710+0000","last_peered":"2024-09-21T22:49:58.986710+0000","last_clean":"2024-09-21T22:49:58.986710+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:49:58.986710+0000","last_undegraded":"2024-09-21T22:49:58.986710+0000","last_fullsized":"2024-09-21T22:49:58.986710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.991381+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:49:58.991381+0000","last_peered":"2024-09-21T22:49:58.991381+0000","last_clean":"2024-09-21T22:49:58.991381+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:49:58.991381+0000","last_undegraded":"2024-09-21T22:49:58.991381+0000","last_fullsized":"2024-09-21T22:49:58.991381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.985930+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:49:58.985930+0000","last_peered":"2024-09-21T22:49:58.985930+0000","last_clean":"2024-09-21T22:49:58.985930+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:49:58.985930+0000","last_undegraded":"2024-09-21T22:49:58.985930+0000","last_fullsized":"2024-09-21T22:49:58.985930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476740,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:50:03.020 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T22:50:03.020 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T22:50:03.020 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T22:50:03.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T22:50:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:03.329 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T22:50:03.346 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T22:50:03.346 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T22:50:03.356 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T22:50:03.356 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T22:50:03.356 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T22:50:03.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T22:50:03.660 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T22:50:03.660 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T22:50:03.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:50:03.935 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:03.935 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T22:50:03.271013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235880,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3063808,"data_stored":1812110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.305356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.207004+0000","last_change":"2024-09-21T22:49:59.207004+0000","last_active":"2024-09-21T22:49:59.207004+0000","last_peered":"2024-09-21T22:49:59.207004+0000","last_clean":"2024-09-21T22:49:59.207004+0000","last_became_active":"2024-09-21T22:49:56.978761+0000","last_became_peered":"2024-09-21T22:49:56.978761+0000","last_unstale":"2024-09-21T22:49:59.207004+0000","last_undegraded":"2024-09-21T22:49:59.207004+0000","last_fullsized":"2024-09-21T22:49:59.207004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:02.238042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068420099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995262+0000","last_change":"2024-09-21T22:49:58.995262+0000","last_active":"2024-09-21T22:49:58.995262+0000","last_peered":"2024-09-21T22:49:58.995262+0000","last_clean":"2024-09-21T22:49:58.995262+0000","last_became_active":"2024-09-21T22:49:56.975265+0000","last_became_peered":"2024-09-21T22:49:56.975265+0000","last_unstale":"2024-09-21T22:49:58.995262+0000","last_undegraded":"2024-09-21T22:49:58.995262+0000","last_fullsized":"2024-09-21T22:49:58.995262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:13:34.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986824+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:49:58.986824+0000","last_peered":"2024-09-21T22:49:58.986824+0000","last_clean":"2024-09-21T22:49:58.986824+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:49:58.986824+0000","last_undegraded":"2024-09-21T22:49:58.986824+0000","last_fullsized":"2024-09-21T22:49:58.986824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995224+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:49:58.995224+0000","last_peered":"2024-09-21T22:49:58.995224+0000","last_clean":"2024-09-21T22:49:58.995224+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:49:58.995224+0000","last_undegraded":"2024-09-21T22:49:58.995224+0000","last_fullsized":"2024-09-21T22:49:58.995224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.992797+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:49:58.992797+0000","last_peered":"2024-09-21T22:49:58.992797+0000","last_clean":"2024-09-21T22:49:58.992797+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:49:58.992797+0000","last_undegraded":"2024-09-21T22:49:58.992797+0000","last_fullsized":"2024-09-21T22:49:58.992797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.203552+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:49:59.203552+0000","last_peered":"2024-09-21T22:49:59.203552+0000","last_clean":"2024-09-21T22:49:59.203552+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:49:59.203552+0000","last_undegraded":"2024-09-21T22:49:59.203552+0000","last_fullsized":"2024-09-21T22:49:59.203552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986710+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:49:58.986710+0000","last_peered":"2024-09-21T22:49:58.986710+0000","last_clean":"2024-09-21T22:49:58.986710+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:49:58.986710+0000","last_undegraded":"2024-09-21T22:49:58.986710+0000","last_fullsized":"2024-09-21T22:49:58.986710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.991381+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:49:58.991381+0000","last_peered":"2024-09-21T22:49:58.991381+0000","last_clean":"2024-09-21T22:49:58.991381+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:49:58.991381+0000","last_undegraded":"2024-09-21T22:49:58.991381+0000","last_fullsized":"2024-09-21T22:49:58.991381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.985930+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:49:58.985930+0000","last_peered":"2024-09-21T22:49:58.985930+0000","last_clean":"2024-09-21T22:49:58.985930+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:49:58.985930+0000","last_undegraded":"2024-09-21T22:49:58.985930+0000","last_fullsized":"2024-09-21T22:49:58.985930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476741,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:50:03.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:50:04.232 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:04.232 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:50:04.249 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T22:50:03.271013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235880,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3063808,"data_stored":1812110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.305356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.207004+0000","last_change":"2024-09-21T22:49:59.207004+0000","last_active":"2024-09-21T22:49:59.207004+0000","last_peered":"2024-09-21T22:49:59.207004+0000","last_clean":"2024-09-21T22:49:59.207004+0000","last_became_active":"2024-09-21T22:49:56.978761+0000","last_became_peered":"2024-09-21T22:49:56.978761+0000","last_unstale":"2024-09-21T22:49:59.207004+0000","last_undegraded":"2024-09-21T22:49:59.207004+0000","last_fullsized":"2024-09-21T22:49:59.207004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:02.238042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068420099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995262+0000","last_change":"2024-09-21T22:49:58.995262+0000","last_active":"2024-09-21T22:49:58.995262+0000","last_peered":"2024-09-21T22:49:58.995262+0000","last_clean":"2024-09-21T22:49:58.995262+0000","last_became_active":"2024-09-21T22:49:56.975265+0000","last_became_peered":"2024-09-21T22:49:56.975265+0000","last_unstale":"2024-09-21T22:49:58.995262+0000","last_undegraded":"2024-09-21T22:49:58.995262+0000","last_fullsized":"2024-09-21T22:49:58.995262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:13:34.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986824+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:49:58.986824+0000","last_peered":"2024-09-21T22:49:58.986824+0000","last_clean":"2024-09-21T22:49:58.986824+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:49:58.986824+0000","last_undegraded":"2024-09-21T22:49:58.986824+0000","last_fullsized":"2024-09-21T22:49:58.986824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.995224+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:49:58.995224+0000","last_peered":"2024-09-21T22:49:58.995224+0000","last_clean":"2024-09-21T22:49:58.995224+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:49:58.995224+0000","last_undegraded":"2024-09-21T22:49:58.995224+0000","last_fullsized":"2024-09-21T22:49:58.995224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.992797+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:49:58.992797+0000","last_peered":"2024-09-21T22:49:58.992797+0000","last_clean":"2024-09-21T22:49:58.992797+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:49:58.992797+0000","last_undegraded":"2024-09-21T22:49:58.992797+0000","last_fullsized":"2024-09-21T22:49:58.992797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:59.203552+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:49:59.203552+0000","last_peered":"2024-09-21T22:49:59.203552+0000","last_clean":"2024-09-21T22:49:59.203552+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:49:59.203552+0000","last_undegraded":"2024-09-21T22:49:59.203552+0000","last_fullsized":"2024-09-21T22:49:59.203552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.986710+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:49:58.986710+0000","last_peered":"2024-09-21T22:49:58.986710+0000","last_clean":"2024-09-21T22:49:58.986710+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:49:58.986710+0000","last_undegraded":"2024-09-21T22:49:58.986710+0000","last_fullsized":"2024-09-21T22:49:58.986710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.991381+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:49:58.991381+0000","last_peered":"2024-09-21T22:49:58.991381+0000","last_clean":"2024-09-21T22:49:58.991381+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:49:58.991381+0000","last_undegraded":"2024-09-21T22:49:58.991381+0000","last_fullsized":"2024-09-21T22:49:58.991381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T22:49:58.985930+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:49:58.985930+0000","last_peered":"2024-09-21T22:49:58.985930+0000","last_clean":"2024-09-21T22:49:58.985930+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:49:58.985930+0000","last_undegraded":"2024-09-21T22:49:58.985930+0000","last_fullsized":"2024-09-21T22:49:58.985930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476741,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":78936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":78917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:50:04.250 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T22:50:04.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 23 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 56013142-0739-4aee-ba29-30eddcc2dbe0 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-21T22:49:36.309693+0000 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-09-21T22:49:58.971136+0000 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 5 2024-09-21T22:50:04.529 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client jewel 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout: removed_snaps_queue [2~1] 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 8 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/2211646084,v1:172.21.15.94:6809/2211646084] [v2:172.21.15.94:6810/2211646084,v1:172.21.15.94:6811/2211646084] exists,up ecfcea04-7198-4c57-bc74-45aaf791e3b7 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/1102146551,v1:172.21.15.94:6825/1102146551] [v2:172.21.15.94:6826/1102146551,v1:172.21.15.94:6827/1102146551] exists,up b6b749e6-4a66-4d8b-bf50-c9634a6927d5 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/2482161893,v1:172.21.15.94:6817/2482161893] [v2:172.21.15.94:6818/2482161893,v1:172.21.15.94:6819/2482161893] exists,up 4e90da65-2fdd-4c85-a207-70d5f22c95a4 2024-09-21T22:50:04.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/2344760438,v1:172.21.15.94:6801/2344760438] [v2:172.21.15.94:6802/2344760438,v1:172.21.15.94:6803/2344760438] exists,up 3c97651a-402c-4171-9648-c615dd754c94 2024-09-21T22:50:04.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6819/1105568802,v1:172.21.15.99:6821/1105568802] [v2:172.21.15.99:6823/1105568802,v1:172.21.15.99:6826/1105568802] exists,up b935b00e-0d91-4f2b-9f2d-2c2be1144ca8 2024-09-21T22:50:04.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 16 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/2691947692,v1:172.21.15.99:6820/2691947692] [v2:172.21.15.99:6822/2691947692,v1:172.21.15.99:6824/2691947692] exists,up 559dea61-5abd-423e-9dfb-cf92f9e6d4a4 2024-09-21T22:50:04.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/545441398,v1:172.21.15.99:6811/545441398] [v2:172.21.15.99:6812/545441398,v1:172.21.15.99:6813/545441398] exists,up aeee922d-53c2-41f4-9433-8a9691f3e674 2024-09-21T22:50:04.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/1713641021,v1:172.21.15.99:6801/1713641021] [v2:172.21.15.99:6802/1713641021,v1:172.21.15.99:6803/1713641021] exists,up b40a3234-6950-4a73-9a5c-f0c086450816 2024-09-21T22:50:04.546 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/2211646084,v1:172.21.15.94:6809/2211646084] [v2:172.21.15.94:6810/2211646084,v1:172.21.15.94:6811/2211646084] exists,up ecfcea04-7198-4c57-bc74-45aaf791e3b7', 'osd.1 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/1102146551,v1:172.21.15.94:6825/1102146551] [v2:172.21.15.94:6826/1102146551,v1:172.21.15.94:6827/1102146551] exists,up b6b749e6-4a66-4d8b-bf50-c9634a6927d5', 'osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/2482161893,v1:172.21.15.94:6817/2482161893] [v2:172.21.15.94:6818/2482161893,v1:172.21.15.94:6819/2482161893] exists,up 4e90da65-2fdd-4c85-a207-70d5f22c95a4', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/2344760438,v1:172.21.15.94:6801/2344760438] [v2:172.21.15.94:6802/2344760438,v1:172.21.15.94:6803/2344760438] exists,up 3c97651a-402c-4171-9648-c615dd754c94', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6819/1105568802,v1:172.21.15.99:6821/1105568802] [v2:172.21.15.99:6823/1105568802,v1:172.21.15.99:6826/1105568802] exists,up b935b00e-0d91-4f2b-9f2d-2c2be1144ca8', 'osd.5 up in weight 1 up_from 16 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/2691947692,v1:172.21.15.99:6820/2691947692] [v2:172.21.15.99:6822/2691947692,v1:172.21.15.99:6824/2691947692] exists,up 559dea61-5abd-423e-9dfb-cf92f9e6d4a4', 'osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/545441398,v1:172.21.15.99:6811/545441398] [v2:172.21.15.99:6812/545441398,v1:172.21.15.99:6813/545441398] exists,up aeee922d-53c2-41f4-9433-8a9691f3e674', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/1713641021,v1:172.21.15.99:6801/1713641021] [v2:172.21.15.99:6802/1713641021,v1:172.21.15.99:6803/1713641021] exists,up b40a3234-6950-4a73-9a5c-f0c086450816'] 2024-09-21T22:50:04.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mon.a", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster": "ceph", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "admin_socket_mode": "", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_client_required": "cephx, none", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_cluster_required": "cephx", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_debug": "true", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T22:50:04.656 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_service_required": "cephx", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_supported": "cephx", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio": "true", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_reap_max": "16", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_async_discard": "false", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_block_size": "4096", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio": "false", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_enable_discard": "false", 2024-09-21T22:50:04.657 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_flock_retry": "3", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_inject_crash": "0", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring": "false", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring_hipri": "false", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_type": "", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T22:50:04.658 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_allocator": "hybrid", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_buffered_io": "true", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T22:50:04.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_replay_recovery": "false", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_sync_write": "false", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_allocator": "hybrid", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T22:50:04.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_create": "true", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_create": "false", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_path": "", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_size": "0", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_path": "", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_size": "107374182400", 2024-09-21T22:50:04.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_create": "false", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_path": "", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs": "true", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_autotune": "true", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T22:50:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size": "0", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_type": "2q", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_clone_cow": "true", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T22:50:04.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_mode": "none", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_freelist": "false", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T22:50:04.664 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T22:50:04.665 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fail_eio": "false", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T22:50:04.666 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T22:50:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size": "0", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T22:50:04.668 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_mem": "512", 2024-09-21T22:50:04.669 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_tracing": "false", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T22:50:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T22:50:04.671 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_cluster_require_version": "2", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_require_signatures": "false", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_require_version": "2", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_service_require_signatures": "false", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_service_require_version": "2", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_sign_messages": "true", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "chdir": "", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_acl_type": "", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_asio_thread_count": "2", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_cache_mid": "0.750000", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_cache_size": "16384", 2024-09-21T22:50:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_caps_release_delay": "5", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_check_pool_perm": "true", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_force_sync_read": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_getattr_caps": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_inject_features": "", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_die_on_failed_remount": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_dirsize_rbytes": "true", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_force_lazyio": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_fs": "", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_inject_release_failure": "false", 2024-09-21T22:50:04.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_max_inline_size": "4096", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mds_namespace": "", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_metadata": "", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_gid": "-1", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_timeout": "300", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_uid": "-1", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mountpoint": "/", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_notify_timeout": "10", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc": "true", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_objects": "1000", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_size": "209715200", 2024-09-21T22:50:04.674 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_permissions": "true", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_quota": "true", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_quota_df": "true", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_max_bytes": "0", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_max_periods": "4", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_min": "131072", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_reconnect_stale": "false", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_shutdown_timeout": "30", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_snapdir": ".snap", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_tick_interval": "1", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_trace": "", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_use_faked_inos": "false", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_use_random_mds": "false", 2024-09-21T22:50:04.675 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog": "false", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog_port": "12201", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_monitors": "default=true", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog": "false", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog_level": "info", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "-", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_network": "", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_network_interface": "", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_isal": "false", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_level": "5", 2024-09-21T22:50:04.676 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zstd_level": "1", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location": "", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location_hook": "", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location_hook_timeout": "10", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "daemonize": "false", 2024-09-21T22:50:04.677 INFO:teuthology.orchestra.run.smithi094.stdout: "daos_pool": "tank", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_alienstore": "0/5", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asok": "1/5", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asok_assert_abort": "false", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_auth": "20/20", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bdev": "1/3", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bluefs": "1/5", 2024-09-21T22:50:04.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bluestore": "1/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_buffer": "0/1", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cephsqlite": "0/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_client": "0/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_compressor": "1/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_context": "0/1", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crush": "1/1", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crypto": "1/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cyanstore": "0/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_dpdk": "1/5", 2024-09-21T22:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_eventtrace": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_filer": "0/1", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_filestore": "1/3", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_finisher": "1/1", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_fuse": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_javaclient": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_journal": "1/3", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_journaler": "0/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_kstore": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_lockdep": "0/1", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mclock": "1/5", 2024-09-21T22:50:04.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_balancer": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_locker": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_log": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_migrator": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_memstore": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mgr": "2/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mgrc": "1/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mon": "20/20", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_monc": "0/10", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_ms": "1/1", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_none": "0/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objclass": "0/5", 2024-09-21T22:50:04.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objectcacher": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objecter": "0/1", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_optracker": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_osd": "1/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_paxos": "20/20", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_perfcounter": "1/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_prioritycache": "1/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rados": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_replay": "0/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_refs": "0/0", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_reserver": "1/1", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw": "1/5", 2024-09-21T22:50:04.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_access": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_flight": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_sync": "1/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rocksdb": "4/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_backref": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_cache": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_device": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_epm": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T22:50:04.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_journal": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_lba": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_odata": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_omap": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_onode": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_t": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_tm": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_striper": "0/1", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_test": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_throttle": "1/1", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_timer": "0/1", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_tp": "0/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_trace": "1/5", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T22:50:04.684 INFO:teuthology.orchestra.run.smithi094.stdout: "device_failure_prediction_mode": "none", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_graylog": "false", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_journald": "false", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_stderr": "true", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_syslog": "false", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event_tracing": "false", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_http_port": "9926", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_prio_limit": "5", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_sort_metrics": "true", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_stats_period": "5", 2024-09-21T22:50:04.685 INFO:teuthology.orchestra.run.smithi094.stdout: "fake_statfs_for_testing": "0", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "fatal_signal_handlers": "true", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filer_max_purge_ops": "10", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filer_max_truncate_ops": "128", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_blackhole": "false", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_btrfs_snap": "true", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_caller_concurrency": "10", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_omap_check": "false", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_verify_split": "false", 2024-09-21T22:50:04.686 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_dump_file": "", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fadvise": "true", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fail_eio": "true", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fd_cache_size": "128", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fiemap": "false", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_inject_stall": "0", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_parallel": "false", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_trailing": "false", 2024-09-21T22:50:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_writeahead": "false", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_kill_at": "0", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T22:50:04.688 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_merge_threshold": "-10", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_odsync_write": "false", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_backend_path": "", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_threads": "2", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_punch_hole": "false", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T22:50:04.689 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_ops": "50", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_seek_data_hole": "false", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_sloppy_crc": "false", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_splice": "false", 2024-09-21T22:50:04.690 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_split_multiple": "2", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_split_rand_factor": "20", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_update_to": "1000", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T22:50:04.691 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_xfs_extsize": "false", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_zfs_snap": "false", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fio_dir": "/tmp/fio", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "56013142-0739-4aee-ba29-30eddcc2dbe0", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_allow_other": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_big_writes": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_debug": "false", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_default_permissions": "false", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_disable_pagecache": "false", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_max_write": "0", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_multithreaded": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_require_active_mds": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_set_user_groups": "true", 2024-09-21T22:50:04.692 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_move": "true", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_read": "true", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_write": "true", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "gss_target_name": "ceph", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_file": "", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_inject_failure": "0", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_interval": "5", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "host": "smithi094", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T22:50:04.693 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "inject_early_sigterm": "false", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "jaeger_agent_port": "6799", 2024-09-21T22:50:04.694 INFO:teuthology.orchestra.run.smithi094.stdout: "jaeger_tracing_enable": "false", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_aio": "true", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_align_min_size": "65536", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_block_align": "true", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_block_size": "4096", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_dio": "true", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_discard": "false", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_force_aio": "false", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_ignore_corruption": "false", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_max_write_entries": "100", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_replay_from": "0", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T22:50:04.695 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_write_header_frequency": "0", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_zero_on_create": "false", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_prefetch_periods": "10", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_prezero_periods": "5", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_write_head_interval": "15", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "keyfile": "", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_backend": "rocksdb", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_max_bytes": "67108864", 2024-09-21T22:50:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_max_ops": "512", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_onode_map_size": "1024", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_sync_transaction": "false", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "librados_thread_count": "2", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "lockdep": "false", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "lockdep_force_backtrace": "false", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_coarse_timestamps": "true", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_flush_on_exit": "false", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_graylog_port": "12201", 2024-09-21T22:50:04.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_max_new": "1000", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_max_recent": "10000", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_stderr_prefix": "", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": "true", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_graylog": "false", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_journald": "false", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_stderr": "false", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_syslog": "false", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_action_on_write_error": "1", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_alternate_name_max": "8192", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_asio_thread_count": "2", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_export_pin": "true", 2024-09-21T22:50:04.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_interval": "10", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_max": "-1", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_max_until": "-1", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_merge_size": "50", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_mode": "0", 2024-09-21T22:50:04.699 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_bits": "3", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_size": "10000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T22:50:04.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_mid": "0.700000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_interval": "1", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T22:50:04.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_auth_pins": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_frag": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_scatterstat": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_subtrees": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_default_dir_hash": "2", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_defer_session_stale": "true", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T22:50:04.702 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_max_entries": "0", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_prefetch": "true", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_early_reply": "true", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_enable_op_tracker": "true", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_enforce_unique_name": "true", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T22:50:04.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_health_dummy": "false", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T22:50:04.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_join_fs": "", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_journal_format": "1", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_create_at": "0", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_export_at": "0", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_import_at": "0", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_at": "0", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T22:50:04.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_link_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_openc_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_rename_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_max_events": "-1", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_max_segments": "128", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_pause": "false", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_segment_size": "0", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T22:50:04.706 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_threshold": "128", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_completed_requests": "100000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_export_size": "20971520", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_file_recover": "32", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_files": "64", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_ops": "8192", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_metrics_update_interval": "2", 2024-09-21T22:50:04.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_min_caps_per_client": "100", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_numa_node": "-1", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_duration": "600", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_size": "20", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_log_threshold": "5", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_ping_grace": "15", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_ping_interval": "5", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T22:50:04.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_caps": "30000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_replay_interval": "1.000000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_root_ino_gid": "0", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_root_ino_uid": "0", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T22:50:04.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T22:50:04.710 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_shutdown_check": "0", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_skip_ino": "0", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_min_uid": "0", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_rstat": "false", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T22:50:04.711 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_symlink_recovery": "true", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_thrash_exports": "0", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_thrash_fragments": "0", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_tick_interval": "5.000000", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_valgrind_exit": "false", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_verify_backtrace": "1", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_verify_scatter": "false", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_wipe_sessions": "false", 2024-09-21T22:50:04.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool_debug": "false", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_page_set": "false", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_page_size": "65536", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_bytes": "134217728", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_messages": "512", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_disabled_modules": "", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T22:50:04.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mds_messages": "128", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mon_messages": "128", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_duration": "600", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_size": "20", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_log_threshold": "5", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T22:50:04.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_osd_messages": "8192", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_pool": "true", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_standby_modules": "true", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_stats_period": "5", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_stats_threshold": "5", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_tick_period": "2", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_allow_pool_delete": "true", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T22:50:04.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_bytes": "104857600", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_target_rank": "-1", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T22:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_start": "false", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_trim": "true", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T22:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cpu_threads": "4", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_avail_crit": "5", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_avail_warn": "5", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_json": "false", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_extra_checks": "false", 2024-09-21T22:50:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_down_uptime_grace": "60", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_election_default_strategy": "1", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_election_timeout": "5.000000", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T22:50:04.719 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_enable_op_tracker": "true", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_fake_pool_delete": "false", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_force_quorum_join": "false", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_log_update_period": "5", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_max_detail": "50", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog": "true", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_host": "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.99", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_host_override": "", 2024-09-21T22:50:04.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_initial_members": "", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease": "5.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_full_interval": "50", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_max": "10000", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_max_summary": "50", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T22:50:04.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_log_epochs": "500", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_osd": "10000", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_memory_autotune": "true", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_memory_target": "2147483648", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T22:50:04.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_digest_period": "5", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_complaint_time": "30", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_duration": "600", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_size": "20", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_log_threshold": "5", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T22:50:04.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_cache_size": "500", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T22:50:04.724 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_report_timeout": "900", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T22:50:04.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T22:50:04.726 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_max_osds": "4", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_interval": "86400", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_max_keys": "100", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_timeout": "300", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_session_timeout": "300", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_smart_report_timeout": "5", 2024-09-21T22:50:04.727 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_pool_size": "4", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_debug": "false", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_tick_interval": "5", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T22:50:04.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T22:50:04.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_older_version": "true", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T22:50:04.730 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": "", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_tracing_enabled": "false", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_op_threads": "3", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_cm": "false", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_device_name": "", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T22:50:04.731 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_sl": "3", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_type": "ib", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_reap_threshold": "5", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_before_connect": "false", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_ipv4": "true", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_ipv6": "false", 2024-09-21T22:50:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_msgr1": "true", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_msgr2": "true", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_port_max": "7568", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_port_min": "6800", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_retry_count": "3", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_retry_delay": "5", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_client": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mds": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mgr": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mon": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_osd": "false", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_client_mode": "crc secure", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_cluster_type": "", 2024-09-21T22:50:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_compress_secure": "false", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_crc_data": "true", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_crc_header": "true", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_bug": "true", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_old_message": "true", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T22:50:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hugepages": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_lro": "true", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_pmd": "", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_port_id": "0", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T22:50:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dump_on_send": "false", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_type": "", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_network_congestion": "0", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_max_accept_failures": "4", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_max_backoff": "15.000000", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T22:50:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compress_mode": "none", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_pq_min_cost": "65536", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_public_type": "", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_service_mode": "crc secure", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_nodelay": "true", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_type": "async+posix", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "no_config_file": "false", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T22:50:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inflight_ops": "1024", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_timeout": "10.000000", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objectstore_blackhole": "false", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "openssl_engine_opts": "", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_max_ops": "4", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T22:50:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_slop": "0.020000", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_asio_thread_count": "2", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_scan_max": "512", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_scan_min": "64", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_peering": "false", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_beacon_report_interval": "300", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_duration": "30", 2024-09-21T22:50:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_blkin_trace_all": "false", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_update_on_start": "true", 2024-09-21T22:50:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_message_cap": "256", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_op_priority": "63", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_watch_timeout": "30", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_command_max_records": "256", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_compact_on_start": "false", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_update_on_start": "true", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T22:50:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_op_order": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T22:50:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_shutdown": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_trim_objects": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T22:50:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_default_notify_timeout": "30", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_enable_op_tracker": "true", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_info": "true", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown": "true", 2024-09-21T22:50:04.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_function_tracing": "false", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_grace": "20", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_interval": "6", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_stale": "600", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T22:50:04.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal_size": "5120", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_kill_backfill_at": "0", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_cache_size": "50", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_dedup": "true", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_message_max": "40", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T22:50:04.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_attr_name_len": "100", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_attr_size": "0", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_backfills": "1", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_markdown_count": "5", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_markdown_period": "600", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_name_len": "2048", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_size": "134217728", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pgls": "1024", 2024-09-21T22:50:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_push_cost": "8388608", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_push_objects": "10", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_scrubs": "3", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_write_size": "90", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T22:50:04.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_profile": "balanced", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_base": "805306368", 2024-09-21T22:50:04.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target": "4294967296", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target_autotune": "false", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_min_recovery_priority": "0", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_report_interval": "5", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T22:50:04.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_num_cache_shards": "32", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_node": "-1", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objecter_finishers": "1", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore": "bluestore", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore_fuse": "false", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore_tracing": "false", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_duration": "600", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_size": "20", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T22:50:04.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_log_threshold": "5", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards": "0", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_thread_timeout": "15", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_open_classes_on_start": "true", 2024-09-21T22:50:04.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_os_flags": "0", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_peering_op_priority": "255", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_delete_priority": "5", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T22:50:04.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_crimson": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T22:50:04.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flags": "0", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_min_size": "0", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_size": "2", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_type": "replicated", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T22:50:04.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_cost": "20971520", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active": "0", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_op_priority": "3", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_priority": "5", 2024-09-21T22:50:04.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_repair_during_recovery": "false", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T22:50:04.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_cost": "52428800", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_end_hour": "0", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T22:50:04.758 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_priority": "5", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_skip_data_digest": "false", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_smart_report_timeout": "5", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_priority": "5", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T22:50:04.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_target_transaction_size": "30", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T22:50:04.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tracing": "false", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_use_stale_snap": "false", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_kill_at": "0", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_max_join_drift": "10", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_min": "500", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_min_wait": "0.050000", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_max": "500", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_min": "250", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_stash_full_interval": "25", 2024-09-21T22:50:04.761 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_trim_max": "500", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_trim_min": "250", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "perf": "true", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "-", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrv": "", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "public_bind_addr": "-", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "public_network": "", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "public_network_interface": "", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_enabled": "false", 2024-09-21T22:50:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_mon_op_timeout": "0", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_osd_op_timeout": "0", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_tracing": "false", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_atime_update_interval": "60", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache": "true", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T22:50:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_policy": "writearound", 2024-09-21T22:50:04.764 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_size": "33554432", 2024-09-21T22:50:04.764 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T22:50:04.764 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T22:50:04.764 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T22:50:04.764 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_compression_hint": "none", 2024-09-21T22:50:04.765 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T22:50:04.765 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T22:50:04.765 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_clone_format": "auto", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_data_pool": "", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_features": "61", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_format": "2", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_map_options": "", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_order": "22", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_pool": "rbd", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_stripe_count": "0", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T22:50:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_io_scheduler": "simple", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_order": "24", 2024-09-21T22:50:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_pool": "", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_splay_width": "4", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T22:50:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T22:50:04.769 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_op_threads": "1", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T22:50:04.770 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_plugins": "", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T22:50:04.771 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T22:50:04.772 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_tracing": "false", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_validate_names": "true", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_validate_pool": "true", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "restapi_base_url": "", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "restapi_log_level": "", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_admin_entry": "admin", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_backend_store": "rados", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_barbican_url": "", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_beast_enable_async": "true", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T22:50:04.773 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_enabled": "true", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_config_store": "rados", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_content_length_compat": "false", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T22:50:04.774 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cross_domain_policy": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T22:50:04.775 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T22:50:04.776 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T22:50:04.777 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_port": "6379", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_window": "30", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T22:50:04.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T22:50:04.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dns_name": "", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dns_s3website_name": "", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_ops_log": "false", 2024-09-21T22:50:04.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_static_website": "false", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_usage_log": "false", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_expose_bucket": "false", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_extended_http_attrs": "", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_filter": "none", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_objs": "32", 2024-09-21T22:50:04.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_init_timeout": "300", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T22:50:04.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_password": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_project": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_token": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_user": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_api_version": "2", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T22:50:04.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_url": "", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_objs": "32", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T22:50:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_worker": "3", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_uri": "", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_http_headers": "", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T22:50:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attr_size": "0", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_listing_results": "1000", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_notify_retries": "10", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T22:50:04.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T22:50:04.787 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_num_control_oids": "8", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_numa_node": "-1", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T22:50:04.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_tracing": "false", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_token": "", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_url": "", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_rados": "false", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T22:50:04.789 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_print_continue": "true", 2024-09-21T22:50:04.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_tracing": "false", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm": "", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm_id": "", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_region": "", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T22:50:04.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_request_uri": "", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_resolve_cname": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_run_sync_thread": "true", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T22:50:04.792 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_script_uri": "", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_service_provider_name": "", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_client_id": "", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_client_secret": "", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_entry": "sts", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_key": "sts", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T22:50:04.793 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_auth_url": "", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_custom_header": "", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_need_stats": "true", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_tenant_name": "", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_url": "", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_lease_period": "120", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T22:50:04.794 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_thread_pool_size": "512", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_comment": "", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_createby": "", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_encoding": "", 2024-09-21T22:50:04.795 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_flag": "false", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_origin": "", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_tracker": "", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_max_shards": "32", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_use_opa_authz": "false", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_counters_cache": "false", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T22:50:04.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_unique_email": "true", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_verify_ssl": "true", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone": "", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone_id": "", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup": "", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup_id": "", 2024-09-21T22:50:04.797 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_block_size": "4096", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T22:50:04.798 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_partition_filters": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_perf": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "run_dir": "/var/run/ceph", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_block_create": "true", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T22:50:04.799 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_device_size": "53687091200", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_full_integrity_check": "false", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_main_device_type": "SSD", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T22:50:04.800 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_segment_size": "67108864", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "set_keepcaps": "false", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "setgroup": "", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "setuser": "", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "setuser_match_path": "", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "thp": "false", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "threadpool_default_timeout": "60", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T22:50:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: "throttler_perf_counter": "true" 2024-09-21T22:50:04.802 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:04.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T22:50:04.943 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a: {} 2024-09-21T22:50:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:50:04.955 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b: {} 2024-09-21T22:50:04.955 INFO:teuthology.orchestra.run.smithi094.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:50:04.965 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c: {} 2024-09-21T22:50:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:50:04.980 DEBUG:teuthology.orchestra.run.smithi094:> type ceph-objectstore-tool 2024-09-21T22:50:04.984 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T22:50:04.984 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-21T22:50:04.989 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T22:50:04.989 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T22:50:05.002 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T22:50:05.003 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T22:50:05.003 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T22:50:05.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:05.003 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T22:50:05.003 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T22:50:05.003 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:50:05.003 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T22:50:05.004 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T22:50:05.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:05.004 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T22:50:05.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:05.005 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T22:50:05.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:50:05.005 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-21T22:50:05.025 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:05.021+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:05.040 INFO:teuthology.orchestra.run.smithi094.stdout:mapped writes DISABLED 2024-09-21T22:50:05.040 INFO:teuthology.orchestra.run.smithi094.stdout:mapped reads DISABLED 2024-09-21T22:50:05.040 INFO:teuthology.orchestra.run.smithi094.stdout:Seed set to 9282 2024-09-21T22:50:05.106 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:05.101+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:05.140 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:05.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:05.140 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:05.140 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:05.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:05.157 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:05.174 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:05.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:05.206 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:05.201+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:05.221 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:05.221 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:05.241 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:05.262 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:05.262 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:05.267 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:05.267 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:05.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:05.329 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:05.325+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:05.408 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:05.401+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:05.434 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:05.434 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:05.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:05.533 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:05.529+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:05.594 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:05.595 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:05.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:05.609 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:05.606+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:05.710 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:05.706+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:05.760 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:05.760 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:05.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:05.817 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:05.813+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:05.886 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:05.886 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:05.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:05.911 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:05.906+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:06.011 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:50:06.011 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:06.006+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:06.069 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.069 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:06.113 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:06.110+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:06.213 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:06.210+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:06.239 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.239 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:06.314 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:06.310+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:06.364 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.364 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:06.425 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:06.421+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:06.495 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.495 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:06.516 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:06.510+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:06.533 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T22:50:06.617 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:06.613+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:06.676 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.676 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:06.717 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:06.714+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:06.818 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:06.814+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:06.856 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:06.856 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:06.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:06.919 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:06.914+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:07.002 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.003 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:07.061 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:07.058+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:07.120 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:07.117+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:07.181 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.181 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:07.221 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:07.217+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:07.312 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.312 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.322 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:07.318+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:07.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:07.424 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:07.417+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:07.492 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.492 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:07.553 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:07.550+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:07.625 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:07.621+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:07.628 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.628 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:07.733 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:07.730+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:07.810 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:07.811 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:07.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:07.869 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:07.866+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:07.927 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:07.925+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:07.992 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:07.992 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:08.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:08.028 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:08.021+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:08.041 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:50:08.124 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:08.124 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:08.124 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:08.124 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:08.124 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:08.129 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:08.126+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:08.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:08.230 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:08.225+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.37092028, 2024-09-21T22:50:08.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:08.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:08.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.251171443, 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013356734, 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055046+0000", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5000000000000001e-07 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055053+0000", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7630000000000003e-06 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055065+0000", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.223e-05 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055072+0000", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4240000000000004e-06 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061686+0000", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066135489999999998 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061927+0000", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000241215 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.062297+0000", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036980999999999998 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.066240+0000", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039434400000000003 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068176+0000", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001935778 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068315+0000", 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000138746 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068402+0000", 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6928999999999994e-05 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:08.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.251126313, 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013602839, 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:08.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055091+0000", 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055184+0000", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2909000000000001e-05 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055198+0000", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055206+0000", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3329999999999997e-06 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.063913+0000", 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087066629999999999 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064150+0000", 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023710199999999999 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064512+0000", 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036199300000000002 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068135+0000", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036232759999999999 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068426+0000", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000290878 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068566+0000", 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000139815 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068693+0000", 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012713000000000001 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.250991934, 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013916583, 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 88 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:08.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055226+0000", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2999999999999999e-07 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055232+0000", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3029999999999996e-06 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055242+0000", 2024-09-21T22:50:08.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0176999999999999e-05 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055249+0000", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067011+0000", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011761939000000001 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067152+0000", 2024-09-21T22:50:08.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140853 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067372+0000", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021964099999999999 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068842+0000", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001470482 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068986+0000", 2024-09-21T22:50:08.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014380299999999999 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069073+0000", 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7498999999999997e-05 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069141+0000", 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7832999999999999e-05 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225335+0000", 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.080881636999999, 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012745968, 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 108 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225335+0000", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225335+0000", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225338+0000", 2024-09-21T22:50:08.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8030000000000001e-06 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225356+0000", 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7802999999999999e-05 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225399+0000", 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3186000000000003e-05 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225451+0000", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2624000000000003e-05 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.232131+0000", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066797840000000002 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.232270+0000", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013854899999999999 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.232567+0000", 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029773400000000002 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.235857+0000", 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032890419999999998 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.237953+0000", 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002096554 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238024+0000", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.059e-05 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238081+0000", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7299000000000001e-05 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:08.339 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.080697362, 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581728, 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 109 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225521+0000", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6360000000000001e-06 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225745+0000", 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022433199999999999 2024-09-21T22:50:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225777+0000", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1955999999999997e-05 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225795+0000", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8244e-05 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233255+0000", 2024-09-21T22:50:08.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074596480000000001 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233518+0000", 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026267499999999997 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233943+0000", 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042513700000000002 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.237933+0000", 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039902160000000004 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241946+0000", 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004013282 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242026+0000", 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9926999999999996e-05 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242101+0000", 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4674999999999999e-05 2024-09-21T22:50:08.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:08.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.080370468, 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019620268, 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:08.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225847+0000", 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3200000000000001e-06 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225858+0000", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0852999999999999e-05 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225876+0000", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7450000000000001e-05 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225885+0000", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6900000000000004e-06 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236393+0000", 2024-09-21T22:50:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010507722000000001 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236628+0000", 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023490800000000001 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236996+0000", 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036759500000000002 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241762+0000", 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047658960000000004 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245313+0000", 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035513490000000001 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245395+0000", 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2372000000000003e-05 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245466+0000", 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1112999999999995e-05 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.080314069, 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020023948999999999, 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7000000000000004e-07 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225910+0000", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6900000000000003e-06 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225931+0000", 2024-09-21T22:50:08.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1364000000000001e-05 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225945+0000", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.402e-05 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238193+0000", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012247655 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238448+0000", 2024-09-21T22:50:08.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025524100000000002 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238841+0000", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000392321 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241921+0000", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030808160000000001 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245516+0000", 2024-09-21T22:50:08.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035948149999999999 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245673+0000", 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015628500000000001 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245926+0000", 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025397199999999999 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.080246059, 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020395262000000001, 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225972+0000", 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.19e-06 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226193+0000", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022107499999999999 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226215+0000", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1742999999999998e-05 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226224+0000", 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4739999999999994e-06 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239713+0000", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013488909 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239935+0000", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022189099999999999 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.240321+0000", 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038573399999999999 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245291+0000", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049707479999999997 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246237+0000", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094542599999999997 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246307+0000", 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0568999999999995e-05 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246366+0000", 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8502999999999999e-05 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.079972068, 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020321847000000001, 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226246+0000", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226254+0000", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226265+0000", 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0293e-05 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226272+0000", 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0169999999999999e-06 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242228+0000", 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01595618 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242480+0000", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025191500000000002 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242848+0000", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036843299999999999 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246075+0000", 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032263650000000001 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246385+0000", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031026200000000001 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246506+0000", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121455 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246566+0000", 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0056999999999998e-05 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.079932425000001, 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020480281999999999, 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226285+0000", 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226291+0000", 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.66e-06 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226302+0000", 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.181e-05 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226310+0000", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.267e-06 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243720+0000", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017410454999999998 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243957+0000", 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023674199999999999 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.244409+0000", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045170900000000001 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246213+0000", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00180459 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246588+0000", 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037527699999999998 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246707+0000", 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118399 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246764+0000", 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7623e-05 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.079870562, 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024358679000000001, 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226347+0000", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8400000000000003e-07 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226468+0000", 2024-09-21T22:50:08.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121322 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226483+0000", 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4569999999999999e-05 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226491+0000", 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0770000000000008e-06 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246868+0000", 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020376894999999999 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246999+0000", 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013165200000000001 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.247197+0000", 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197309 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250432+0000", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032354889999999998 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250549+0000", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011647899999999999 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250638+0000", 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9341999999999999e-05 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250705+0000", 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6660000000000002e-05 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.079708979999999, 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024433862000000001, 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226509+0000", 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9400000000000004e-07 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226515+0000", 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3389999999999998e-06 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226525+0000", 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9070000000000002e-06 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226532+0000", 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.866e-06 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248383+0000", 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021851272000000001 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248545+0000", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016223600000000001 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248767+0000", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222165 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250529+0000", 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001761372 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250736+0000", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000207293 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250864+0000", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000127992 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250941+0000", 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7526000000000003e-05 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.0796735, 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024955143999999999, 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226544+0000", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226550+0000", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226560+0000", 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3270000000000007e-06 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226566+0000", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249453+0000", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022886468 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249587+0000", 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133702 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249882+0000", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029489099999999998 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250965+0000", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010834309999999999 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251227+0000", 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000262522 2024-09-21T22:50:08.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251386+0000", 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158405 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251498+0000", 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011236899999999999 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.582768313000001, 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424575, 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723449+0000", 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8799999999999998e-07 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723452+0000", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.864e-06 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723457+0000", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723461+0000", 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4989999999999999e-06 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733711+0000", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010250377999999999 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733854+0000", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014302600000000001 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.734075+0000", 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000221282 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735173+0000", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010972250000000001 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737475+0000", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023023990000000001 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737557+0000", 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2229000000000002e-05 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737694+0000", 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136816 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.582751224000001, 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01520267, 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723466+0000", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3999999999999998e-07 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723468+0000", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4310000000000001e-06 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723472+0000", 2024-09-21T22:50:08.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7299999999999999e-06 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723475+0000", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9679999999999998e-06 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735404+0000", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011928970000000001 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735473+0000", 2024-09-21T22:50:08.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9604000000000006e-05 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735582+0000", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108411 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738444+0000", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028617740000000001 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738467+0000", 2024-09-21T22:50:08.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3923000000000001e-05 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738556+0000", 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8955999999999993e-05 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738668+0000", 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111663 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.582737550999999, 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015433831, 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723480+0000", 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5400000000000002e-07 2024-09-21T22:50:08.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723516+0000", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6826999999999998e-05 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723529+0000", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2544000000000001e-05 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723544+0000", 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4688999999999999e-05 2024-09-21T22:50:08.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735951+0000", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012407110000000001 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736089+0000", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137916 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736299+0000", 2024-09-21T22:50:08.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209929 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738732+0000", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024338570000000002 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738762+0000", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9442999999999999e-05 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738839+0000", 2024-09-21T22:50:08.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7052999999999996e-05 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738913+0000", 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4009000000000002e-05 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.582663891999999, 2024-09-21T22:50:08.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015612934, 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8599999999999998e-07 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723557+0000", 2024-09-21T22:50:08.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.177e-06 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723562+0000", 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9030000000000003e-06 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723566+0000", 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737789+0000", 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014223372999999999 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737904+0000", 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011495200000000001 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738063+0000", 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158331 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738952+0000", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00088979599999999999 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738987+0000", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4440000000000002e-05 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739077+0000", 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0549000000000003e-05 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739166+0000", 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8200000000000003e-05 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.912077123, 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013813499999999999, 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 158 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:08.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394140+0000", 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1699999999999997e-07 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394254+0000", 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113719 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394274+0000", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0086e-05 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394288+0000", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3917000000000001e-05 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401006+0000", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067183939999999999 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401140+0000", 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013358900000000001 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401345+0000", 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020530799999999999 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403313+0000", 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019673379999999999 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407738+0000", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00442531 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407837+0000", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9029000000000003e-05 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407953+0000", 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116093 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.911902151, 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013836113000000001, 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-21T22:50:08.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394316+0000", 2024-09-21T22:50:08.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.266e-06 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394326+0000", 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0227000000000001e-05 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394341+0000", 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5373e-05 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:08.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394353+0000", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2044e-05 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403365+0000", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090115799999999999 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403506+0000", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140712 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:08.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403703+0000", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197431 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.405003+0000", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00129959 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407975+0000", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029723610000000002 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:08.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408045+0000", 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0253000000000007e-05 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408151+0000", 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105276 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:08.425 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:08.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:08.427 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:08.421+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:08.528 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:08.526+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:08.551 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:08.551 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:08.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:08.628 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:08.622+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:08.686 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:08.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:08.729 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:08.726+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:08.822 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:08.823 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:08.830 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:08.830+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:08.836 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:08.931 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:08.930+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:08.957 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:08.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:50:09.031 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:09.026+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.093 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:09.157 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:09.153+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:09.223 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.223 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.234 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:09.230+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:09.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:09.335 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:09.330+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:09.402 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.402 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:09.461 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:09.457+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:09.532 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.532 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.536 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:09.529+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:09.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:09.637 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:09.633+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:09.712 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.713 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:09.738 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:09.734+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:09.839 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:09.838+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:09.882 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:09.882 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:09.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:09.939 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:09.938+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:10.013 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.013 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:10.073 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:10.069+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:10.141 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:10.137+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:10.147 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.147 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:10.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:10.174 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:10.174 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:10.242 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:10.237+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:10.297 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.297 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:10.357 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:10.354+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:10.440 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.440 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.444 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:10.438+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:10.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:10.545 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:10.541+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:10.619 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.619 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:10.646 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:10.641+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:10.747 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:10.741+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:10.750 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.751 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:10.848 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:10.841+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:10.882 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:10.882 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:10.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:10.947 INFO:teuthology.orchestra.run.smithi094.stdout:skipping zero size read 2024-09-21T22:50:10.947 INFO:teuthology.orchestra.run.smithi094.stdout:skipping zero size read 2024-09-21T22:50:10.948 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:10.946+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:10.966 INFO:teuthology.orchestra.run.smithi094.stdout:4 writesame 0x816304c thru 0x8168126 data_size 0x1(0x50db bytes) 2024-09-21T22:50:10.966 INFO:teuthology.orchestra.run.smithi094.stdout:5 read 0x2e4909e thru 0x2e4af6c (0x1ecf bytes) 2024-09-21T22:50:10.966 INFO:teuthology.orchestra.run.smithi094.stdout:6 read 0x38756e5 thru 0x387cb22 (0x743e bytes) 2024-09-21T22:50:10.966 INFO:teuthology.orchestra.run.smithi094.stdout:7 write 0x59fd0ec thru 0x59fd262 (0x177 bytes) 2024-09-21T22:50:10.970 INFO:teuthology.orchestra.run.smithi094.stdout:8 write 0x6052cae thru 0x6060ef6 (0xe249 bytes) 2024-09-21T22:50:10.976 INFO:teuthology.orchestra.run.smithi094.stdout:9 read 0x758bec2 thru 0x75905ad (0x46ec bytes) 2024-09-21T22:50:10.977 INFO:teuthology.orchestra.run.smithi094.stdout:10 writesame 0xab18f3 thru 0xac0af7 data_size 0x35(0xf205 bytes) 2024-09-21T22:50:11.022 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.022 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.036 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:11.050 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:11.045+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:11.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:50:11.105 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T22:50:11.105 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-21T22:50:11.105 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:50:11.106 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-21T22:50:11.133 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T22:50:11.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-21T22:50:11.150 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:11.145+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:11.183 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.184 INFO:teuthology.orchestra.run.smithi094.stdout:12 punch from 0x5abeb18 to 0x5ac9193, (0xa67b bytes) 2024-09-21T22:50:11.184 INFO:teuthology.orchestra.run.smithi094.stdout:13 writesame 0x1d3aa7d thru 0x1d4092e data_size 0x2e(0x5eb2 bytes) 2024-09-21T22:50:11.189 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0x35cc53 2024-09-21T22:50:11.200 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.200 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.200 INFO:teuthology.orchestra.run.smithi094.stdout:14 trunc from 0x8168127 to 0x35cc53 2024-09-21T22:50:11.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:11.226 INFO:teuthology.orchestra.run.smithi094.stdout:16 write 0xe2be4d thru 0xe37a03 (0xbbb7 bytes) 2024-09-21T22:50:11.229 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.230 INFO:teuthology.orchestra.run.smithi094.stdout:18 punch from 0x5e9c44 to 0x5ebf56, (0x2312 bytes) 2024-09-21T22:50:11.230 INFO:teuthology.orchestra.run.smithi094.stdout:19 punch from 0xb94f3c to 0xba4209, (0xf2cd bytes) 2024-09-21T22:50:11.230 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0x19c9430 2024-09-21T22:50:11.230 INFO:teuthology.orchestra.run.smithi094.stdout:20 trunc from 0xe37a04 to 0x19c9430 2024-09-21T22:50:11.236 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.251 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:11.245+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:11.261 INFO:teuthology.orchestra.run.smithi094.stdout:22 write 0xbb81d99 thru 0xbb843a2 (0x260a bytes) 2024-09-21T22:50:11.265 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.265 INFO:teuthology.orchestra.run.smithi094.stdout:24 read 0x3a03652 thru 0x3a10eb7 (0xd866 bytes) 2024-09-21T22:50:11.265 INFO:teuthology.orchestra.run.smithi094.stdout:26 punch from 0x7271b25 to 0x727c773, (0xac4e bytes) 2024-09-21T22:50:11.265 INFO:teuthology.orchestra.run.smithi094.stdout:27 read 0x43e89f1 thru 0x43edb27 (0x5137 bytes) 2024-09-21T22:50:11.265 INFO:teuthology.orchestra.run.smithi094.stdout:28 writesame 0xbf5143 thru 0xc01f7f data_size 0x1(0xce3d bytes) 2024-09-21T22:50:11.309 INFO:teuthology.orchestra.run.smithi094.stdout:30 read 0x390ca23 thru 0x3913b3a (0x7118 bytes) 2024-09-21T22:50:11.309 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0x3358414 2024-09-21T22:50:11.320 INFO:teuthology.orchestra.run.smithi094.stdout:31 trunc from 0xbb843a3 to 0x3358414 2024-09-21T22:50:11.334 INFO:teuthology.orchestra.run.smithi094.stdout:32 read 0x16d9ef0 thru 0x16e1894 (0x79a5 bytes) 2024-09-21T22:50:11.334 INFO:teuthology.orchestra.run.smithi094.stdout:34 writesame 0xcc0e5f thru 0xcc3c54 data_size 0x4a(0x2df6 bytes) 2024-09-21T22:50:11.337 INFO:teuthology.orchestra.run.smithi094.stdout:35 read 0x370033 thru 0x37e918 (0xe8e6 bytes) 2024-09-21T22:50:11.352 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:11.346+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:11.359 INFO:teuthology.orchestra.run.smithi094.stdout:36 writesame 0xa932eb8 thru 0xa93a13c data_size 0x1(0x7285 bytes) 2024-09-21T22:50:11.359 INFO:teuthology.orchestra.run.smithi094.stdout:37 read 0x8c3b581 thru 0x8c499e3 (0xe463 bytes) 2024-09-21T22:50:11.359 INFO:teuthology.orchestra.run.smithi094.stdout:38 read 0x8f9038d thru 0x8f9a006 (0x9c7a bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:39 read 0x925a784 thru 0x925f82e (0x50ab bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:40 read 0x4bdca53 thru 0x4bdecd3 (0x2281 bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:42 punch from 0x2158d44 to 0x2163b9b, (0xae57 bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:43 read 0x2bfbb3c thru 0x2c062e2 (0xa7a7 bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:44 read 0x984b4bc thru 0x984dc3d (0x2782 bytes) 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:47 write 0x5ac8440 thru 0x5ad5d49 (0xd90a bytes) 2024-09-21T22:50:11.365 INFO:teuthology.orchestra.run.smithi094.stdout:49 read 0xa04add8 thru 0xa058d48 (0xdf71 bytes) 2024-09-21T22:50:11.365 INFO:teuthology.orchestra.run.smithi094.stdout:51 read 0x1431a36 thru 0x1432072 (0x63d bytes) 2024-09-21T22:50:11.365 INFO:teuthology.orchestra.run.smithi094.stdout:52 writesame 0x2cc91bc thru 0x2ccfcbd data_size 0x1(0x6b02 bytes) 2024-09-21T22:50:11.379 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.379 INFO:teuthology.orchestra.run.smithi094.stdout:55 write 0x1e1f075 thru 0x1e2e410 (0xf39c bytes) 2024-09-21T22:50:11.382 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.383 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.385 INFO:teuthology.orchestra.run.smithi094.stdout:56 punch from 0xe4c89f to 0xe55a85, (0x91e6 bytes) 2024-09-21T22:50:11.385 INFO:teuthology.orchestra.run.smithi094.stdout:57 write 0x86bed86 thru 0x86c082e (0x1aa9 bytes) 2024-09-21T22:50:11.390 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.390 INFO:teuthology.orchestra.run.smithi094.stdout:59 write 0x88b34be thru 0x88b76bb (0x41fe bytes) 2024-09-21T22:50:11.393 INFO:teuthology.orchestra.run.smithi094.stdout:60 write 0x9c3eb2e thru 0x9c440df (0x55b2 bytes) 2024-09-21T22:50:11.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:11.399 INFO:teuthology.orchestra.run.smithi094.stdout:61 read 0x609c77d thru 0x60a0172 (0x39f6 bytes) 2024-09-21T22:50:11.417 INFO:teuthology.orchestra.run.smithi094.stdout:63 write 0xed8fe58 thru 0xed9e659 (0xe802 bytes) 2024-09-21T22:50:11.423 INFO:teuthology.orchestra.run.smithi094.stdout:64 write 0x9159502 thru 0x915d6e5 (0x41e4 bytes) 2024-09-21T22:50:11.427 INFO:teuthology.orchestra.run.smithi094.stdout:65 write 0x52ed7c1 thru 0x52ede6f (0x6af bytes) 2024-09-21T22:50:11.432 INFO:teuthology.orchestra.run.smithi094.stdout:67 read 0xaff7af6 thru 0xafff139 (0x7644 bytes) 2024-09-21T22:50:11.432 INFO:teuthology.orchestra.run.smithi094.stdout:68 read 0xaa04385 thru 0xaa0dc02 (0x987e bytes) 2024-09-21T22:50:11.432 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:11.432 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xab2f85e 2024-09-21T22:50:11.441 INFO:teuthology.orchestra.run.smithi094.stdout:70 trunc from 0xed9e65a to 0xab2f85e 2024-09-21T22:50:11.453 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:11.449+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:11.456 INFO:teuthology.orchestra.run.smithi094.stdout:71 read 0x7b774b thru 0x7bd9bc (0x6272 bytes) 2024-09-21T22:50:11.456 INFO:teuthology.orchestra.run.smithi094.stdout:72 write 0x9ad55b8 thru 0x9adc71a (0x7163 bytes) 2024-09-21T22:50:11.460 INFO:teuthology.orchestra.run.smithi094.stdout:73 read 0x3fa1a49 thru 0x3fa8c68 (0x7220 bytes) 2024-09-21T22:50:11.461 INFO:teuthology.orchestra.run.smithi094.stdout:74 clone 1 order 19 su 8192 sc 13 2024-09-21T22:50:11.554 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:11.549+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:11.567 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.567 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:11.654 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:11.649+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:11.706 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.707 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:11.756 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:11.750+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:11.857 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:11.853+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:11.888 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:11.889 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:11.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:11.989 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:11.985+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:50:12.058 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:12.053+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:12.065 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:12.065 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:12.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:12.159 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:12.154+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:12.175 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:12.250 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:12.250 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:12.260 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:12.254+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:12.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:12.279 INFO:teuthology.orchestra.run.smithi094.stdout:76 write 0x6444752 thru 0x6448b42 (0x43f1 bytes) 2024-09-21T22:50:12.295 INFO:teuthology.orchestra.run.smithi094.stdout:77 read 0x90d2fab thru 0x90d4ff8 (0x204e bytes) 2024-09-21T22:50:12.297 INFO:teuthology.orchestra.run.smithi094.stdout:78 writesame 0x7a89405 thru 0x7a939e8 data_size 0x6(0xa5e4 bytes) 2024-09-21T22:50:12.314 INFO:teuthology.orchestra.run.smithi094.stdout:79 read 0x9d9314d thru 0x9d9f70c (0xc5c0 bytes) 2024-09-21T22:50:12.317 INFO:teuthology.orchestra.run.smithi094.stdout:80 punch from 0x9d21c05 to 0x9d22f73, (0x136e bytes) 2024-09-21T22:50:12.325 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:12.334 INFO:teuthology.orchestra.run.smithi094.stdout:81 trunc from 0xab2f85e to 0x427b45 2024-09-21T22:50:12.348 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:12.402 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:12.402 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:12.403 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:12.403 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:12.403 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:83 write 0xd14459d thru 0xd146836 (0x229a bytes) 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:84 read 0xbbc5eb5 thru 0xbbcda93 (0x7bdf bytes) 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:85 read 0x10b46d3 thru 0x10c0888 (0xc1b6 bytes) 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:86 read 0x359d1c8 thru 0x35a8482 (0xb2bb bytes) 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.404 INFO:teuthology.orchestra.run.smithi094.stdout:90 write 0xc569ce0 thru 0xc56bfb9 (0x22da bytes) 2024-09-21T22:50:12.405 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:12.405 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:12.406 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:12.401+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:12.407 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.407 INFO:teuthology.orchestra.run.smithi094.stdout:94 writesame 0x41faf22 thru 0x4203e3a data_size 0x1(0x8f19 bytes) 2024-09-21T22:50:12.417 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:12.419 INFO:teuthology.orchestra.run.smithi094.stdout:95 write 0x19ffd12 thru 0x1a0cd74 (0xd063 bytes) 2024-09-21T22:50:12.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:12.433 INFO:teuthology.orchestra.run.smithi094.stdout:98 read 0x77bd105 thru 0x77c5d96 (0x8c92 bytes) 2024-09-21T22:50:12.433 INFO:teuthology.orchestra.run.smithi094.stdout:100 punch from 0x31fb758 to 0x3203bdc, (0x8484 bytes) 2024-09-21T22:50:12.433 INFO:teuthology.orchestra.run.smithi094.stdout:101 punch from 0x2abece2 to 0x2ace119, (0xf437 bytes) 2024-09-21T22:50:12.433 INFO:teuthology.orchestra.run.smithi094.stdout:104 writesame 0x7a1a82d thru 0x7a1c8a2 data_size 0xf(0x2076 bytes) 2024-09-21T22:50:12.438 INFO:teuthology.orchestra.run.smithi094.stdout:106 read 0x61816d2 thru 0x6189cfe (0x862d bytes) 2024-09-21T22:50:12.438 INFO:teuthology.orchestra.run.smithi094.stdout:107 read 0x950da71 thru 0x9513843 (0x5dd3 bytes) 2024-09-21T22:50:12.438 INFO:teuthology.orchestra.run.smithi094.stdout:109 writesame 0x4452694 thru 0x445f77d data_size 0x8f(0xd0ea bytes) 2024-09-21T22:50:12.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:12.440 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:12.440 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:12.445 INFO:teuthology.orchestra.run.smithi094.stdout:111 writesame 0x57672e2 thru 0x576c82b data_size 0x9(0x554a bytes) 2024-09-21T22:50:12.453 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xdaf21d4 2024-09-21T22:50:12.454 INFO:teuthology.orchestra.run.smithi094.stdout:112 trunc from 0xd146837 to 0xdaf21d4 2024-09-21T22:50:12.460 INFO:teuthology.orchestra.run.smithi094.stdout:113 punch from 0x77b5ba1 to 0x77c14c7, (0xb926 bytes) 2024-09-21T22:50:12.461 INFO:teuthology.orchestra.run.smithi094.stdout:114 read 0x989d0eb thru 0x98a6275 (0x918b bytes) 2024-09-21T22:50:12.461 INFO:teuthology.orchestra.run.smithi094.stdout:115 writesame 0x37ebb5f thru 0x37f4ca0 data_size 0x1(0x9142 bytes) 2024-09-21T22:50:12.462 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:12.462 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:12.465 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:12.466 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:12.477 INFO:teuthology.orchestra.run.smithi094.stdout:116 read 0x82e0cb4 thru 0x82e6583 (0x58d0 bytes) 2024-09-21T22:50:12.477 INFO:teuthology.orchestra.run.smithi094.stdout:117 writesame 0xcbcd71 thru 0xcc36b8 data_size 0xc(0x6948 bytes) 2024-09-21T22:50:12.483 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.483 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.483 INFO:teuthology.orchestra.run.smithi094.stdout:122 punch from 0x4d132e to 0x4d5a23, (0x46f5 bytes) 2024-09-21T22:50:12.483 INFO:teuthology.orchestra.run.smithi094.stdout:125 read 0x55b88da thru 0x55c803b (0xf762 bytes) 2024-09-21T22:50:12.484 INFO:teuthology.orchestra.run.smithi094.stdout:126 read 0x6f21d99 thru 0x6f2b193 (0x93fb bytes) 2024-09-21T22:50:12.496 INFO:teuthology.orchestra.run.smithi094.stdout:128 trunc from 0xdaf21d4 to 0x60c5060 2024-09-21T22:50:12.506 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:12.501+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:12.510 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xdd067db 2024-09-21T22:50:12.521 INFO:teuthology.orchestra.run.smithi094.stdout:129 trunc from 0x60c5060 to 0xdd067db 2024-09-21T22:50:12.526 INFO:teuthology.orchestra.run.smithi094.stdout:130 write 0xcd978ed thru 0xcda4d94 (0xd4a8 bytes) 2024-09-21T22:50:12.539 INFO:teuthology.orchestra.run.smithi094.stdout:131 punch from 0x4a2a0d2 to 0x4a2a68e, (0x5bc bytes) 2024-09-21T22:50:12.539 INFO:teuthology.orchestra.run.smithi094.stdout:132 read 0x490775e thru 0x4911766 (0xa009 bytes) 2024-09-21T22:50:12.539 INFO:teuthology.orchestra.run.smithi094.stdout:133 write 0x560068a thru 0x5602509 (0x1e80 bytes) 2024-09-21T22:50:12.545 INFO:teuthology.orchestra.run.smithi094.stdout:135 read 0xc2adcf9 thru 0xc2b2aaa (0x4db2 bytes) 2024-09-21T22:50:12.545 INFO:teuthology.orchestra.run.smithi094.stdout:136 write 0x45f9255 thru 0x4604bea (0xb996 bytes) 2024-09-21T22:50:12.564 INFO:teuthology.orchestra.run.smithi094.stdout:137 trunc from 0xdd067db to 0x9fba034 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4269.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:49:57.982001+0000", 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.592944857999999, 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002319148, 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982001+0000", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982001+0000", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982001+0000", 2024-09-21T22:50:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982017+0000", 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5426000000000001e-05 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982042+0000", 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5015999999999999e-05 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982061+0000", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9647e-05 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982199+0000", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013776899999999999 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.982255+0000", 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6035999999999997e-05 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.984176+0000", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019211320000000001 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:57.984320+0000", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014345199999999999 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4269.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:49:58.990822+0000", 2024-09-21T22:50:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.584123332000001, 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015189719999999999, 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990822+0000", 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990822+0000", 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990823+0000", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4000000000000002e-07 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990838+0000", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4949999999999999e-05 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990874+0000", 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6263000000000002e-05 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.990899+0000", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5116000000000001e-05 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.991056+0000", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015727499999999999 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.991087+0000", 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1068000000000001e-05 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.992265+0000", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00117752 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:58.992341+0000", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6240000000000002e-05 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:349 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 335872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.358289+0000", 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.21665648200000001, 2024-09-21T22:50:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045817330000000002, 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 349 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358289+0000", 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358289+0000", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358291+0000", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4710000000000001e-06 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358301+0000", 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1649999999999995e-06 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358326+0000", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5265999999999998e-05 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358339+0000", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3166999999999999e-05 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358427+0000", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7972000000000004e-05 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.358434+0000", 2024-09-21T22:50:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2509999999999997e-06 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.360612+0000", 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002177881 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.360732+0000", 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119803 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.360929+0000", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019685500000000001 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.361543+0000", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061397299999999997 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362770+0000", 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001227007 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362830+0000", 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0491000000000001e-05 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362871+0000", 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0431e-05 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:349 3.19 e28/24 ondisk, result = 0)", 2024-09-21T22:50:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.362637+0000", 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.212308573, 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000241024, 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362637+0000", 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362637+0000", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362638+0000", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362643+0000", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362660+0000", 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7139e-05 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362668+0000", 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7300000000000005e-06 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362756+0000", 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7626000000000004e-05 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362764+0000", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.337e-06 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.362878+0000", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114009 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:358 3.4 3:23a25f62:::rbd_data.111ecbf88771.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.364986+0000", 2024-09-21T22:50:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.209959332, 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016295369999999999, 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 358 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.364986+0000", 2024-09-21T22:50:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.364986+0000", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.364987+0000", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.283e-06 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.364993+0000", 2024-09-21T22:50:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7819999999999999e-06 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.365011+0000", 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7350000000000002e-05 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.365020+0000", 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8050000000000001e-06 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.365088+0000", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7719999999999995e-05 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.365241+0000", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015274900000000001 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.365448+0000", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206856 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366198+0000", 2024-09-21T22:50:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074996899999999996 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366509+0000", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031086900000000002 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366571+0000", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2269999999999998e-05 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366616+0000", 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4883999999999999e-05 2024-09-21T22:50:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:358 3.4 e28/24 ondisk, result = 0)", 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.366422+0000", 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.20852388299999999, 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000208901, 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366422+0000", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366422+0000", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366422+0000", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1999999999999998e-07 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366426+0000", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7740000000000002e-06 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366437+0000", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0812000000000001e-05 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366442+0000", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2140000000000004e-06 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366495+0000", 2024-09-21T22:50:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2623000000000001e-05 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366503+0000", 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7549999999999994e-06 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.366630+0000", 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012790300000000001 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:405 3.6 e28/24 3:635c6e2c:::rbd_data.111ecbf88771.0000000000000035:head v 28'1, mlcod=0'0)", 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.423623+0000", 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.151322502, 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001048916, 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423623+0000", 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423623+0000", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423623+0000", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2399999999999999e-07 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423649+0000", 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5873000000000001e-05 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423662+0000", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2604e-05 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423671+0000", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3409999999999992e-06 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423801+0000", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129534 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.423821+0000", 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9935000000000001e-05 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.424616+0000", 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079566000000000001 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.424672+0000", 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5645e-05 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:410 3.8 e28/24 3:1454f85c:::rbd_data.111ecbf88771.000000000000003a:head v 28'4, mlcod=27'2)", 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.429987+0000", 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.14495810000000001, 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094455799999999996, 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.429987+0000", 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.429987+0000", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.429988+0000", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1119999999999999e-06 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430001+0000", 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2279999999999999e-05 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430016+0000", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5559000000000001e-05 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430025+0000", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4619999999999996e-06 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430092+0000", 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7662000000000004e-05 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430115+0000", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.281e-05 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430883+0000", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076736700000000003 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.430932+0000", 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9305999999999999e-05 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:431 3.6 e28/24 3:65aa58d7:::rbd_data.111ecbf88771.00000000000000b2:head v 28'2, mlcod=0'0)", 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.450415+0000", 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.124530826, 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00099281400000000007, 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450415+0000", 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450415+0000", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450416+0000", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3430000000000001e-06 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450463+0000", 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7237999999999997e-05 2024-09-21T22:50:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450483+0000", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200999999999999e-05 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450494+0000", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0246e-05 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450566+0000", 2024-09-21T22:50:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2655000000000001e-05 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.450596+0000", 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9215000000000001e-05 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.451358+0000", 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076249900000000003 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.451407+0000", 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9416999999999998e-05 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:449 3.0 3:04e5c2a8:::rbd_data.111ecbf88771.0000000000000070:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.108874393, 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010416138, 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 449 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466074+0000", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7240000000000001e-06 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466093+0000", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9432999999999998e-05 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466112+0000", 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8777e-05 2024-09-21T22:50:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466121+0000", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8699999999999998e-06 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466198+0000", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7168000000000004e-05 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466348+0000", 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150359 2024-09-21T22:50:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.467428+0000", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010794089999999999 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476341+0000", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0089129889999999996 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476418+0000", 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6756999999999997e-05 2024-09-21T22:50:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476450+0000", 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2313e-05 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476487+0000", 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7339000000000003e-05 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:450 3.0 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.108733702, 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010348049999999999, 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 450 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6400000000000002e-07 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466217+0000", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1169999999999997e-06 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466230+0000", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3653e-05 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466242+0000", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1479e-05 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472115+0000", 2024-09-21T22:50:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058731429999999999 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472225+0000", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110074 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472818+0000", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059322200000000002 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476407+0000", 2024-09-21T22:50:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035886849999999999 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476497+0000", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0493000000000002e-05 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476536+0000", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8464999999999998e-05 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476560+0000", 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4054999999999999e-05 2024-09-21T22:50:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:449 3.0 e28/24 ondisk, result = 0)", 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.105598858, 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070061350000000001, 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5899999999999995e-07 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469351+0000", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469362+0000", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0709e-05 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469369+0000", 2024-09-21T22:50:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7839999999999998e-06 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476323+0000", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0069546770000000003 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476333+0000", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9620000000000005e-06 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476353+0000", 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9544000000000001e-05 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:451 3.1a e28/24 3:58b09410:::rbd_data.111ecbf88771.0000000000000072:head v 28'3, mlcod=26'1)", 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.471045+0000", 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.103900361, 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013660129999999999, 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471045+0000", 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471045+0000", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471046+0000", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-07 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471393+0000", 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000347308 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471404+0000", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1304999999999999e-05 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471410+0000", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5430000000000001e-06 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471463+0000", 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3624e-05 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.471481+0000", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7204e-05 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472355+0000", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00087399000000000005 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472411+0000", 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6440999999999997e-05 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:450 3.0 e28/24 ondisk, result = 0)", 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.474970+0000", 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.099975906000000003, 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001592544, 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474970+0000", 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474970+0000", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474970+0000", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5600000000000005e-07 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474978+0000", 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8599999999999993e-06 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474991+0000", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2976000000000001e-05 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474999+0000", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4249999999999998e-06 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476386+0000", 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001386789 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476393+0000", 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9990000000000002e-06 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476562+0000", 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168839 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:459 3.8 e28/24 3:155672b4:::rbd_data.111ecbf88771.0000000000000013:head v 28'5, mlcod=27'3)", 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.480549+0000", 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.094396044999999998, 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011161789999999999, 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480549+0000", 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480549+0000", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480550+0000", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200000000000001e-07 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480565+0000", 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4701999999999999e-05 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480583+0000", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7876e-05 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480593+0000", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0029e-05 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480862+0000", 2024-09-21T22:50:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026949299999999999 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.480879+0000", 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6646e-05 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.481633+0000", 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075405399999999999 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.481666+0000", 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2466999999999999e-05 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:498 3.8 e28/24 3:17181f91:::rbd_data.111ecbf88771.0000000000000196:head v 28'6, mlcod=28'4)", 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.529971+0000", 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.044974256999999997, 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001118757, 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.529971+0000", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.529971+0000", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.529972+0000", 2024-09-21T22:50:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2810000000000001e-06 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.529984+0000", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878999999999999e-05 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.530006+0000", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1576000000000001e-05 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.530016+0000", 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407999999999999e-05 2024-09-21T22:50:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.530098+0000", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1179000000000003e-05 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.530125+0000", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7937e-05 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.531036+0000", 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091058800000000004 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.531090+0000", 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3909000000000002e-05 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:504 3.6 e28/24 3:64e5b7ad:::rbd_data.111ecbf88771.000000000000019c:head v 28'3, mlcod=28'1)", 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.534368+0000", 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.040577126999999998, 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084240300000000003, 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534368+0000", 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534368+0000", 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534370+0000", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1790000000000001e-06 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534384+0000", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4800999999999999e-05 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534401+0000", 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6986999999999998e-05 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534412+0000", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1022000000000001e-05 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534476+0000", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3924000000000007e-05 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.534502+0000", 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5690999999999998e-05 2024-09-21T22:50:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.535159+0000", 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065687000000000004 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.535211+0000", 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1928999999999997e-05 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:520 3.8 e28/24 3:17181f91:::rbd_data.111ecbf88771.0000000000000196:head v 28'7, mlcod=28'5)", 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.567194+0000", 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.007751918, 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081875100000000005, 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567194+0000", 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567194+0000", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567195+0000", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567205+0000", 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0509999999999999e-05 2024-09-21T22:50:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567228+0000", 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2175999999999999e-05 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567237+0000", 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6069999999999997e-06 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567299+0000", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2248000000000004e-05 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567320+0000", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567956+0000", 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000635707 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.568012+0000", 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6320000000000003e-05 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:526 3.6 e28/24 3:64e5b7ad:::rbd_data.111ecbf88771.000000000000019c:head v 28'4, mlcod=28'2)", 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.567277+0000", 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0076686330000000002, 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010019639999999999, 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567277+0000", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567277+0000", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567278+0000", 2024-09-21T22:50:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567286+0000", 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7970000000000001e-06 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567296+0000", 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0498e-05 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567303+0000", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4420000000000003e-06 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567368+0000", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5067000000000006e-05 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.567386+0000", 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8142999999999999e-05 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.568231+0000", 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084515600000000001 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.568279+0000", 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7868999999999999e-05 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:12.661 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:12.662 INFO:teuthology.orchestra.run.smithi094.stdout:138 punch from 0x6b1075d to 0x6b1efe6, (0xe889 bytes) 2024-09-21T22:50:12.662 INFO:teuthology.orchestra.run.smithi094.stdout:139 write 0x88f2b82 thru 0x88f7dc6 (0x5245 bytes) 2024-09-21T22:50:12.662 INFO:teuthology.orchestra.run.smithi094.stdout:140 trunc from 0x9fba034 to 0xaa51466 2024-09-21T22:50:12.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:143 read 0x8de9e9a thru 0x8df03a3 (0x650a bytes) 2024-09-21T22:50:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:144 trunc from 0xaa51466 to 0xa931e7e 2024-09-21T22:50:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:145 read 0x136baae thru 0x1377333 (0xb886 bytes) 2024-09-21T22:50:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xe969419 2024-09-21T22:50:12.665 INFO:teuthology.orchestra.run.smithi094.stdout:147 trunc from 0xa931e7e to 0xe969419 2024-09-21T22:50:12.665 INFO:teuthology.orchestra.run.smithi094.stdout:148 read 0x68f18b1 thru 0x68f6100 (0x4850 bytes) 2024-09-21T22:50:12.665 INFO:teuthology.orchestra.run.smithi094.stdout:149 punch from 0x99452b4 to 0x994c234, (0x6f80 bytes) 2024-09-21T22:50:12.665 INFO:teuthology.orchestra.run.smithi094.stdout:150 trunc from 0xe969419 to 0x452ac05 2024-09-21T22:50:12.665 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:12.658+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:12.673 INFO:teuthology.orchestra.run.smithi094.stdout:152 write 0x978bd2f thru 0x979a2be (0xe590 bytes) 2024-09-21T22:50:12.707 INFO:teuthology.orchestra.run.smithi094.stdout:154 write 0xd5a324c thru 0xd5a985e (0x6613 bytes) 2024-09-21T22:50:12.716 INFO:teuthology.orchestra.run.smithi094.stdout:155 writesame 0x7dc2815 thru 0x7dcadc9 data_size 0x1(0x85b5 bytes) 2024-09-21T22:50:12.731 INFO:teuthology.orchestra.run.smithi094.stdout:157 write 0xab8baf thru 0xabb5d8 (0x2a2a bytes) 2024-09-21T22:50:12.734 INFO:teuthology.orchestra.run.smithi094.stdout:158 read 0x745096e thru 0x745fc7a (0xf30d bytes) 2024-09-21T22:50:12.734 INFO:teuthology.orchestra.run.smithi094.stdout:159 read 0xa5d18d8 thru 0xa5d9fd5 (0x86fe bytes) 2024-09-21T22:50:12.734 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.734 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.734 INFO:teuthology.orchestra.run.smithi094.stdout:164 write 0xd406ad7 thru 0xd4123c2 (0xb8ec bytes) 2024-09-21T22:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:165 write 0x2637475 thru 0x2646083 (0xec0f bytes) 2024-09-21T22:50:12.761 INFO:teuthology.orchestra.run.smithi094.stdout:166 read 0x56f5ee3 thru 0x56fa5f8 (0x4716 bytes) 2024-09-21T22:50:12.762 INFO:teuthology.orchestra.run.smithi094.stdout:168 read 0xa53ac84 thru 0xa53e4d1 (0x384e bytes) 2024-09-21T22:50:12.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T22:50:12.762 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:12.757+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:12.768 INFO:teuthology.orchestra.run.smithi094.stdout:169 write 0xd9a652e thru 0xd9b31dd (0xccb0 bytes) 2024-09-21T22:50:12.782 INFO:teuthology.orchestra.run.smithi094.stdout:170 read 0xaa3cbbc thru 0xaa4372c (0x6b71 bytes) 2024-09-21T22:50:12.782 INFO:teuthology.orchestra.run.smithi094.stdout:171 punch from 0xa83c309 to 0xa842abe, (0x67b5 bytes) 2024-09-21T22:50:12.782 INFO:teuthology.orchestra.run.smithi094.stdout:172 writesame 0x576a219 thru 0x576d27a data_size 0xb(0x3062 bytes) 2024-09-21T22:50:12.787 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.793 INFO:teuthology.orchestra.run.smithi094.stdout:176 trunc from 0xd9b31de to 0x923e6e8 2024-09-21T22:50:12.809 INFO:teuthology.orchestra.run.smithi094.stdout:178 write 0x266dcd1 thru 0x2675d5d (0x808d bytes) 2024-09-21T22:50:12.814 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:12.815 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:12.824 INFO:teuthology.orchestra.run.smithi094.stdout:180 write 0xdeb5cea thru 0xdec065b (0xa972 bytes) 2024-09-21T22:50:12.826 INFO:teuthology.orchestra.run.smithi099.stdout:2.7 2024-09-21T22:50:12.826 INFO:teuthology.orchestra.run.smithi099.stdout:2.6 2024-09-21T22:50:12.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:12.835 INFO:teuthology.orchestra.run.smithi094.stdout:181 punch from 0x6987be2 to 0x698f755, (0x7b73 bytes) 2024-09-21T22:50:12.835 INFO:teuthology.orchestra.run.smithi094.stdout:183 writesame 0x78c4769 thru 0x78c4f26 data_size 0x1(0x7be bytes) 2024-09-21T22:50:12.838 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.839 INFO:teuthology.orchestra.run.smithi094.stdout:188 write 0x16642e8 thru 0x167030f (0xc028 bytes) 2024-09-21T22:50:12.847 INFO:teuthology.orchestra.run.smithi094.stdout:190 write 0xba1bf10 thru 0xba27937 (0xba28 bytes) 2024-09-21T22:50:12.859 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.859 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.864 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:12.857+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:12.865 INFO:teuthology.orchestra.run.smithi094.stdout:194 write 0xeae8951 thru 0xeaf230f (0x99bf bytes) 2024-09-21T22:50:12.887 INFO:teuthology.orchestra.run.smithi094.stdout:195 trunc from 0xeaf2310 to 0x3a95ed7 2024-09-21T22:50:12.917 INFO:teuthology.orchestra.run.smithi094.stdout:198 read 0x3471eec thru 0x347361f (0x1734 bytes) 2024-09-21T22:50:12.917 INFO:teuthology.orchestra.run.smithi094.stdout:199 read 0x1a3b8a5 thru 0x1a4877e (0xceda bytes) 2024-09-21T22:50:12.927 INFO:teuthology.orchestra.run.smithi094.stdout:200 write 0x6c3c7a9 thru 0x6c47bbc (0xb414 bytes) 2024-09-21T22:50:12.941 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:12.942 INFO:teuthology.orchestra.run.smithi094.stdout:203 punch from 0x56ab33 to 0x573abe, (0x8f8b bytes) 2024-09-21T22:50:12.945 INFO:teuthology.orchestra.run.smithi094.stdout:204 trunc from 0x6c47bbd to 0xa22558c 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:207 punch from 0x2983526 to 0x2987df2, (0x48cc bytes) 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:208 read 0x79b0a99 thru 0x79b6429 (0x5991 bytes) 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:209 read 0x97fd8e9 thru 0x97ffd79 (0x2491 bytes) 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:210 read 0x63ac38e thru 0x63b332a (0x6f9d bytes) 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:212 read 0x2ae0af0 thru 0x2ae9907 (0x8e18 bytes) 2024-09-21T22:50:12.951 INFO:teuthology.orchestra.run.smithi094.stdout:213 clone 2 order 18 su 16384 sc 11 2024-09-21T22:50:12.958 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:12.959 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:12.965 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:12.961+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:12.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:13.066 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:13.061+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:13.143 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4269.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:49:56.979158+0000", 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.160288318999999, 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011494560000000001, 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979158+0000", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979158+0000", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979158+0000", 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.34e-07 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979168+0000", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3619999999999995e-06 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979193+0000", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5831999999999999e-05 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979206+0000", 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.258e-05 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979253+0000", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6555999999999999e-05 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.979280+0000", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7843000000000001e-05 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.980275+0000", 2024-09-21T22:50:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00099497400000000003 2024-09-21T22:50:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:49:56.980307+0000", 2024-09-21T22:50:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1875000000000002e-05 2024-09-21T22:50:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:449 3.0 e28/24 3:04e5c2a8:::rbd_data.111ecbf88771.0000000000000070:head v 28'1, mlcod=0'0)", 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.468368+0000", 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.67107777700000004, 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089514, 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468368+0000", 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468368+0000", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468370+0000", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.345e-06 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468388+0000", 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7971000000000001e-05 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468411+0000", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3717e-05 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468419+0000", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4379999999999998e-06 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468488+0000", 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8717e-05 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.468512+0000", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4272000000000001e-05 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469227+0000", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071559800000000002 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469263+0000", 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6081999999999998e-05 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.153 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:450 3.0 e28/24 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head v 28'2, mlcod=0'0)", 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.473916+0000", 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.665529959, 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010115219999999999, 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473916+0000", 2024-09-21T22:50:13.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473916+0000", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473917+0000", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473944+0000", 2024-09-21T22:50:13.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6652000000000002e-05 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473961+0000", 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6674999999999999e-05 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.473974+0000", 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3271000000000001e-05 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474038+0000", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4146000000000006e-05 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474064+0000", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6154000000000001e-05 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474884+0000", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081957999999999998 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.474928+0000", 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3914e-05 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:629 3.0 e28/24 3:06f0f6f2:::rbd_data.111ecbf88771.0000000000000133:head v 28'3, mlcod=28'1)", 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.688435+0000", 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.45101124100000001, 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091017300000000002, 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.158 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688435+0000", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688435+0000", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688436+0000", 2024-09-21T22:50:13.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.407e-06 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688452+0000", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6184e-05 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688471+0000", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8797e-05 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688484+0000", 2024-09-21T22:50:13.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2464e-05 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688559+0000", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5369000000000003e-05 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.688587+0000", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7871000000000001e-05 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.689286+0000", 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000699057 2024-09-21T22:50:13.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.689345+0000", 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9024000000000003e-05 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:630 3.0 e28/24 3:07ca066a:::rbd_data.111ecbf88771.0000000000000134:head v 28'4, mlcod=28'2)", 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.692753+0000", 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.44669309899999998, 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094435500000000002, 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692753+0000", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692753+0000", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692754+0000", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2500000000000004e-07 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692762+0000", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5410000000000003e-06 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692776+0000", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4012000000000001e-05 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692783+0000", 2024-09-21T22:50:13.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0140000000000001e-06 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692864+0000", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1649000000000004e-05 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692892+0000", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7528000000000001e-05 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.693620+0000", 2024-09-21T22:50:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072847899999999998 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.693697+0000", 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7206999999999994e-05 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:647 3.4 e28/24 3:25dc7d81:::rbd_data.111ecbf88771.00000000000001a9:head v 28'5, mlcod=27'3)", 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.714678+0000", 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.42476852599999998, 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00087497300000000003, 2024-09-21T22:50:13.166 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714678+0000", 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714678+0000", 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714679+0000", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.852e-06 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714714+0000", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4276999999999997e-05 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714739+0000", 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5168e-05 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714751+0000", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1894e-05 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714835+0000", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4302999999999996e-05 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.714863+0000", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8274000000000001e-05 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.715516+0000", 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065257300000000004 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.715553+0000", 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6631999999999998e-05 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:656 3.0 e28/24 3:06e36901:::rbd_data.111ecbf88771.00000000000000fd:head v 28'5, mlcod=28'3)", 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.726141+0000", 2024-09-21T22:50:13.170 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.41330509999999998, 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086420900000000001, 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726141+0000", 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726141+0000", 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726142+0000", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726166+0000", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726182+0000", 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6032000000000002e-05 2024-09-21T22:50:13.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726193+0000", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1361e-05 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726251+0000", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7803000000000002e-05 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726277+0000", 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5737000000000001e-05 2024-09-21T22:50:13.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.726971+0000", 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000694726 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.727005+0000", 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3905999999999997e-05 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.174 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:663 3.4 e28/24 3:25dc7d81:::rbd_data.111ecbf88771.00000000000001a9:head v 28'6, mlcod=28'4)", 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.734917+0000", 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.40452946200000001, 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012505769999999999, 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.734917+0000", 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.734917+0000", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.734919+0000", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2979999999999999e-06 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.734965+0000", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5804999999999999e-05 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.734995+0000", 2024-09-21T22:50:13.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.735010+0000", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4678000000000001e-05 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.735091+0000", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1218999999999996e-05 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.735114+0000", 2024-09-21T22:50:13.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2955999999999998e-05 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.736128+0000", 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010138460000000001 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.736167+0000", 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9405e-05 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:682 3.0 e28/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 28'6, mlcod=28'4)", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.750218+0000", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.38922806799999998, 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011402210000000001, 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750218+0000", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750218+0000", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750220+0000", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.584e-06 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750246+0000", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6621000000000001e-05 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750268+0000", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1670000000000001e-05 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750279+0000", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0723000000000001e-05 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750346+0000", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7448999999999997e-05 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.750371+0000", 2024-09-21T22:50:13.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4399000000000001e-05 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.751317+0000", 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094604300000000004 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.751358+0000", 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1732e-05 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:710 3.4 e28/24 3:21749e99:::rbd_data.111ecbf88771.00000000000001b1:head v 28'7, mlcod=28'5)", 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.779875+0000", 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.35957128199999999, 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011761549999999999, 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779875+0000", 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779875+0000", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779877+0000", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6500000000000001e-06 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779923+0000", 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6721999999999998e-05 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779946+0000", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779958+0000", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1647e-05 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780036+0000", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8461000000000007e-05 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780061+0000", 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5011000000000002e-05 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780991+0000", 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092977900000000004 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.781051+0000", 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9735000000000003e-05 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:711 3.0 e28/24 3:079e4dd3:::rbd_data.111ecbf88771.00000000000001b2:head v 28'7, mlcod=28'5)", 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.35945038899999998, 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013689850000000001, 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780009+0000", 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2391e-05 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780016+0000", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.126e-06 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780022+0000", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2299999999999996e-06 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780096+0000", 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3664999999999994e-05 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780122+0000", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6571999999999999e-05 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.781333+0000", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001211057 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.781365+0000", 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1439000000000002e-05 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:725 3.0 e28/24 3:06f0f6f2:::rbd_data.111ecbf88771.0000000000000133:head v 28'8, mlcod=28'6)", 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.796357+0000", 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.34308929799999999, 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012382770000000001, 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796357+0000", 2024-09-21T22:50:13.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796357+0000", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796358+0000", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.514e-06 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796377+0000", 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8617000000000001e-05 2024-09-21T22:50:13.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796405+0000", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8512000000000001e-05 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796424+0000", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8291000000000002e-05 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796496+0000", 2024-09-21T22:50:13.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1970000000000004e-05 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.796521+0000", 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5462000000000001e-05 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797545+0000", 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001023527 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797595+0000", 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0383999999999997e-05 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:726 3.0 e28/24 3:07ca066a:::rbd_data.111ecbf88771.0000000000000134:head v 28'9, mlcod=28'6)", 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.34242343600000003, 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016183619999999999, 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797024+0000", 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0599999999999999e-07 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797030+0000", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4969999999999998e-06 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797044+0000", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3868e-05 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797052+0000", 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1869999999999997e-06 2024-09-21T22:50:13.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797110+0000", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7995000000000003e-05 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797129+0000", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9344e-05 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798601+0000", 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014715419999999999 2024-09-21T22:50:13.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798641+0000", 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0022999999999999e-05 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:732 3.4 e28/24 3:25dc7d81:::rbd_data.111ecbf88771.00000000000001a9:head v 28'8, mlcod=28'6)", 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.798760+0000", 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.34068570199999998, 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084075700000000003, 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798760+0000", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798760+0000", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798761+0000", 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798770+0000", 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4570000000000005e-06 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798788+0000", 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7926000000000001e-05 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798796+0000", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5979999999999997e-06 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798872+0000", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5840999999999994e-05 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798901+0000", 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8841000000000001e-05 2024-09-21T22:50:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.799537+0000", 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063635799999999995 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.799601+0000", 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3751999999999995e-05 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.203 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:740 3.4 e28/24 3:21749e99:::rbd_data.111ecbf88771.00000000000001b1:head v 28'9, mlcod=28'7)", 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.800344+0000", 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.33910244099999998, 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082200999999999997, 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800344+0000", 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800344+0000", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800344+0000", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.54e-07 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800350+0000", 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0100000000000001e-06 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800366+0000", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5512e-05 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800374+0000", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6170000000000001e-06 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800453+0000", 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9833999999999995e-05 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800477+0000", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4037999999999999e-05 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801125+0000", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064787199999999997 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801166+0000", 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0373000000000001e-05 2024-09-21T22:50:13.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:741 3.0 e28/24 3:079e4dd3:::rbd_data.111ecbf88771.00000000000001b2:head v 28'10, mlcod=28'8)", 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.800920+0000", 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.33852611199999999, 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094125600000000002, 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800920+0000", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800920+0000", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800921+0000", 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3499999999999995e-07 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800927+0000", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9370000000000001e-06 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800937+0000", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0786e-05 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.800944+0000", 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6699999999999997e-06 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801029+0000", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4550999999999997e-05 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801053+0000", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4527000000000001e-05 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801819+0000", 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076564000000000005 2024-09-21T22:50:13.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.801861+0000", 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2410000000000002e-05 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:753 3.0 e28/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 28'11, mlcod=28'9)", 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.810358+0000", 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.32908801300000001, 2024-09-21T22:50:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093479299999999995, 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810358+0000", 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810358+0000", 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810359+0000", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9299999999999996e-07 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810367+0000", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810384+0000", 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7625000000000002e-05 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810392+0000", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0160000000000008e-06 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810452+0000", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9314999999999997e-05 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.810475+0000", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3328000000000001e-05 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.811252+0000", 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00077703199999999998 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.811293+0000", 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0806000000000002e-05 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:849 3.0 e28/24 3:06e36901:::rbd_data.111ecbf88771.00000000000000fd:head v 28'12, mlcod=28'10)", 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.894858+0000", 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.24458823800000001, 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00077995899999999999, 2024-09-21T22:50:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894858+0000", 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894858+0000", 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894859+0000", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-06 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894868+0000", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4759999999999998e-06 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894885+0000", 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7700999999999999e-05 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894895+0000", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894973+0000", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8009000000000005e-05 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.894993+0000", 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0636e-05 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.895588+0000", 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059466599999999999 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.895638+0000", 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9908999999999999e-05 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:901 3.9 e28/24 3:94f6a430:::rbd_data.111ecbf88771.00000000000000d1:head v 28'3, mlcod=27'1)", 2024-09-21T22:50:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.937102+0000", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.20234434100000001, 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086712699999999998, 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937102+0000", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937102+0000", 2024-09-21T22:50:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937103+0000", 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937117+0000", 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3319000000000001e-05 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937132+0000", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5362e-05 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937139+0000", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3159999999999999e-06 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937190+0000", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0478e-05 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937207+0000", 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7418000000000002e-05 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937923+0000", 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000715239 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.937969+0000", 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6402000000000001e-05 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.225 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:13.225 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone1 from 0xa22558c (overlap 0x427b45) to 0x27e38f 2024-09-21T22:50:13.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:13.227 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:13.222+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:13.302 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T22:50:13.327 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:13.322+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:13.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 2.7 --file /var/log/ceph/exp.2.7.6 2024-09-21T22:50:13.402 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:13.402 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:13.402 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:13.402 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:13.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:13.465 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:13.461+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:13.530 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:13.525+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:13.585 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:13.585 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:13.585 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:13.585 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:13.585 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:13.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:217 punch from 0x761b5fb to 0x762b17e, (0xfb83 bytes) 2024-09-21T22:50:13.612 INFO:teuthology.orchestra.run.smithi094.stdout:219 write 0x4a686d5 thru 0x4a6cb40 (0x446c bytes) 2024-09-21T22:50:13.622 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:13.629 INFO:teuthology.orchestra.run.smithi094.stdout:221 trunc from 0xa22558c to 0x5336242 2024-09-21T22:50:13.629 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:13.625+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:13.676 INFO:teuthology.orchestra.run.smithi094.stdout:222 trunc from 0x5336242 to 0x5c48310 2024-09-21T22:50:13.706 INFO:teuthology.orchestra.run.smithi094.stdout:223 write 0x7eac178 thru 0x7eb192c (0x57b5 bytes) 2024-09-21T22:50:13.717 INFO:teuthology.orchestra.run.smithi094.stdout:225 write 0x674e5f6 thru 0x674ea98 (0x4a3 bytes) 2024-09-21T22:50:13.722 INFO:teuthology.orchestra.run.smithi094.stdout:226 punch from 0x5a5b15f to 0x5a5b176, (0x17 bytes) 2024-09-21T22:50:13.722 INFO:teuthology.orchestra.run.smithi094.stdout:227 write 0x2f18b21 thru 0x2f2261c (0x9afc bytes) 2024-09-21T22:50:13.730 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:13.726+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:615 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.0607303020000001, 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011471453, 2024-09-21T22:50:13.737 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 615 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5199999999999996e-07 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673353+0000", 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.864e-06 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673362+0000", 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6640000000000004e-06 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673367+0000", 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7400000000000004e-06 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.676392+0000", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030256549999999999 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.676572+0000", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017976900000000001 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.677171+0000", 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059889800000000005 2024-09-21T22:50:13.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680874+0000", 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037028819999999998 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684627+0000", 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037529310000000001 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684720+0000", 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3356000000000005e-05 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684821+0000", 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101042 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:616 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.743 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.060691788, 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012626187000000001, 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 616 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673389+0000", 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5400000000000001e-07 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673391+0000", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7180000000000001e-06 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673400+0000", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6959999999999994e-06 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673405+0000", 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8439999999999997e-06 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.677910+0000", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045045809999999997 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.678042+0000", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013247500000000001 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.678602+0000", 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00055957299999999995 2024-09-21T22:50:13.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684502+0000", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059001110000000004 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685933+0000", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001431444 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685970+0000", 2024-09-21T22:50:13.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6600999999999997e-05 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686014+0000", 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4589999999999998e-05 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:617 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.0606675240000001, 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012746733999999999, 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 617 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673413+0000", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000005e-07 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673416+0000", 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5900000000000002e-06 2024-09-21T22:50:13.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673423+0000", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7360000000000001e-06 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673428+0000", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5190000000000003e-06 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.679721+0000", 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0062929830000000003 2024-09-21T22:50:13.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.679840+0000", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011886 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680378+0000", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053876300000000005 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686031+0000", 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056522059999999999 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686076+0000", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5603999999999999e-05 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686115+0000", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9156000000000003e-05 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686159+0000", 2024-09-21T22:50:13.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3832999999999999e-05 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:618 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.060645909, 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012878603000000001, 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 618 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673435+0000", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.46e-07 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673437+0000", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.52e-06 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673445+0000", 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3599999999999996e-06 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673450+0000", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.634e-06 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680924+0000", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074742860000000001 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.681001+0000", 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7169999999999995e-05 2024-09-21T22:50:13.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.681342+0000", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000340281 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686038+0000", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046963689999999997 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686207+0000", 2024-09-21T22:50:13.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168868 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686269+0000", 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2445000000000003e-05 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686313+0000", 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3223999999999998e-05 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.760 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:619 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.060623665, 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013060782, 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 619 2024-09-21T22:50:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673457+0000", 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.58e-07 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673459+0000", 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6539999999999999e-06 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673465+0000", 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.02e-06 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673470+0000", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4479999999999996e-06 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682015+0000", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085449849999999997 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682193+0000", 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017775199999999999 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.683339+0000", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011463549999999999 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686337+0000", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00299797 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686383+0000", 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6462999999999999e-05 2024-09-21T22:50:13.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686467+0000", 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3796999999999997e-05 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686517+0000", 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9879999999999997e-05 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:620 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.0606042170000001, 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014257994, 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 620 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.767 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:13.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673479+0000", 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5560000000000002e-06 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673495+0000", 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.615e-05 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673500+0000", 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9439999999999996e-06 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684881+0000", 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011381242 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684990+0000", 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109197 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685289+0000", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029849100000000001 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687532+0000", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022436510000000002 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687588+0000", 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5655999999999997e-05 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687652+0000", 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3544e-05 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687734+0000", 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2126999999999995e-05 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:621 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.772 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.060573553, 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018613351, 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 621 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673507+0000", 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673510+0000", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8059999999999999e-06 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673516+0000", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3099999999999997e-06 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673520+0000", 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4619999999999998e-06 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686563+0000", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013042876 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686663+0000", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9570999999999999e-05 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687059+0000", 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039618800000000001 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687883+0000", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082348899999999995 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692061+0000", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004178022 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692094+0000", 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3473999999999998e-05 2024-09-21T22:50:13.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692120+0000", 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5711000000000002e-05 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:679 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746163+0000", 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.98791694600000002, 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011067230000000001, 2024-09-21T22:50:13.778 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 679 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746163+0000", 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746163+0000", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746163+0000", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746166+0000", 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.548e-06 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746172+0000", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0220000000000001e-06 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746177+0000", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.752572+0000", 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063956439999999998 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.752644+0000", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1159000000000001e-05 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.752945+0000", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030093200000000001 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754446+0000", 2024-09-21T22:50:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015016509999999999 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.757162+0000", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027154879999999998 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.757200+0000", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8402999999999998e-05 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.757230+0000", 2024-09-21T22:50:13.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0151e-05 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:680 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.98789718999999998, 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013778161000000001, 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 680 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746187+0000", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9160000000000003e-06 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746193+0000", 2024-09-21T22:50:13.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746200+0000", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5760000000000002e-06 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753332+0000", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071319840000000001 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753409+0000", 2024-09-21T22:50:13.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7111000000000001e-05 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753698+0000", 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028885599999999999 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754451+0000", 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075351400000000005 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759904+0000", 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054529679999999999 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759937+0000", 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3018999999999997e-05 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759961+0000", 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3889000000000001e-05 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:705 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.768539+0000", 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.96554061400000002, 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011008555, 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 705 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768539+0000", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768539+0000", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768540+0000", 2024-09-21T22:50:13.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768544+0000", 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6219999999999998e-06 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768564+0000", 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0789999999999999e-05 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.768573+0000", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6880000000000005e-06 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.775168+0000", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065946959999999997 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.775263+0000", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4854999999999999e-05 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.775652+0000", 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00038962099999999999 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.776331+0000", 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067871200000000004 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779468+0000", 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031372470000000001 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779505+0000", 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7082000000000002e-05 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779548+0000", 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2620000000000002e-05 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1045 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.795 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55084710800000003, 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011667999, 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1045 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:13.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183234+0000", 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2300000000000001e-06 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183241+0000", 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4699999999999999e-06 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183259+0000", 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8133000000000001e-05 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183285+0000", 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6083000000000001e-05 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.189608+0000", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063234750000000003 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.189839+0000", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230881 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.190878+0000", 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001038509 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194705+0000", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038270959999999999 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194730+0000", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5534e-05 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194809+0000", 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8337000000000006e-05 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194901+0000", 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2250999999999995e-05 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1046 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55077753900000004, 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015794288, 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1046 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183304+0000", 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1930000000000001e-06 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183311+0000", 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3339999999999996e-06 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183339+0000", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8030000000000001e-05 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183346+0000", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.192825+0000", 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094795499999999998 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193017+0000", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000192287 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193797+0000", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000779388 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197391+0000", 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035944969999999999 2024-09-21T22:50:13.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198914+0000", 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015223039999999999 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199027+0000", 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113127 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199097+0000", 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0024999999999994e-05 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1047 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55072491400000001, 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017348148000000001, 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1047 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:13.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183356+0000", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183362+0000", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.806e-06 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183371+0000", 2024-09-21T22:50:13.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5529999999999998e-06 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183378+0000", 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8689999999999998e-06 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194941+0000", 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011562767999999999 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195056+0000", 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115281 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195493+0000", 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043665799999999998 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199120+0000", 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036272919999999998 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200347+0000", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001226782 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200566+0000", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021947500000000001 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200703+0000", 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013704200000000001 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1048 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55069258899999995, 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017543792999999998, 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1048 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183388+0000", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183392+0000", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7139999999999999e-06 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183401+0000", 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2469999999999997e-06 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183408+0000", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0469999999999996e-06 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195923+0000", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012514658 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196021+0000", 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8113000000000001e-05 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196394+0000", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037272199999999997 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200738+0000", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043443179999999998 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200803+0000", 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5166000000000006e-05 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200869+0000", 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6190000000000002e-05 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200931+0000", 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1996999999999996e-05 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1049 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.818 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55066295399999998, 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017955736999999999, 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1049 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183418+0000", 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183421+0000", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5580000000000001e-06 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183431+0000", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3430000000000002e-06 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183438+0000", 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-21T22:50:13.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197463+0000", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014025892999999999 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197596+0000", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013223600000000001 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198242+0000", 2024-09-21T22:50:13.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064663999999999998 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201081+0000", 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002839144 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201120+0000", 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8563000000000003e-05 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201262+0000", 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014153000000000001 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201373+0000", 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111236 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1050 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:13.824 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55063309400000005, 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020333185, 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1050 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183448+0000", 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183451+0000", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6389999999999998e-06 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183460+0000", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183467+0000", 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9600000000000003e-06 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199154+0000", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015686734000000001 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199259+0000", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105364 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199748+0000", 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048820500000000002 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203483+0000", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003735787 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203519+0000", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5954000000000002e-05 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203653+0000", 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013328799999999999 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203780+0000", 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012739399999999999 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1051 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.550579712, 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021843896000000002, 2024-09-21T22:50:13.830 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1051 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183501+0000", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2399999999999997e-07 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183506+0000", 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-21T22:50:13.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183527+0000", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1353e-05 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183535+0000", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0180000000000001e-06 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201414+0000", 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01787888 2024-09-21T22:50:13.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201496+0000", 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2201999999999997e-05 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201824+0000", 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032739100000000001 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205098+0000", 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032737280000000001 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205130+0000", 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2805000000000002e-05 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205258+0000", 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127911 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205344+0000", 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5882000000000002e-05 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1052 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55053188600000003, 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022039640999999999, 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1052 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183549+0000", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183553+0000", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1130000000000003e-06 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183563+0000", 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6930000000000002e-06 2024-09-21T22:50:13.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183570+0000", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202302+0000", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018731863000000001 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202413+0000", 2024-09-21T22:50:13.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111117 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202922+0000", 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050897300000000002 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205382+0000", 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024604240000000001 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205403+0000", 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1058999999999999e-05 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205503+0000", 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100068 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205588+0000", 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4584000000000002e-05 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1053 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.55050079600000001, 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022880587000000001, 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1053 2024-09-21T22:50:13.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183580+0000", 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183592+0000", 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.224e-05 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183602+0000", 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094e-05 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183609+0000", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1810000000000004e-06 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203845+0000", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020235907000000001 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203896+0000", 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1187999999999998e-05 2024-09-21T22:50:13.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204112+0000", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021558999999999999 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205609+0000", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001496743 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206370+0000", 2024-09-21T22:50:13.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076130800000000004 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206421+0000", 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0491000000000002e-05 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206460+0000", 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9218999999999998e-05 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1054 3.2 3:46641ad1:::rbd_data.111ecbf88771.0000000000000000:head [stat,truncate 204800] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.54630398499999999, 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019736885999999999, 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1054 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:13.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187777+0000", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:13.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187783+0000", 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2780000000000001e-06 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187794+0000", 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1464999999999999e-05 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187811+0000", 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6716999999999999e-05 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204451+0000", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016639856000000001 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204504+0000", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3763e-05 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204604+0000", 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9433999999999997e-05 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205621+0000", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010175049999999999 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207340+0000", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718884 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207409+0000", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8959000000000002e-05 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:13.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207513+0000", 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010366600000000001 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:13.853 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:13.854 INFO:teuthology.orchestra.run.smithi094.stdout:229 trunc from 0x7eb192d to 0x5b3fc6b 2024-09-21T22:50:13.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:13.855 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:13.856 INFO:teuthology.orchestra.run.smithi094.stdout:234 trunc from 0x5b3fc6b to 0x99409ca 2024-09-21T22:50:13.856 INFO:teuthology.orchestra.run.smithi094.stdout:235 write 0x188b175 thru 0x189383e (0x86ca bytes) 2024-09-21T22:50:13.856 INFO:teuthology.orchestra.run.smithi094.stdout:236 writesame 0x734df5c thru 0x7357eb2 data_size 0x1(0x9f57 bytes) 2024-09-21T22:50:13.856 INFO:teuthology.orchestra.run.smithi094.stdout:238 write 0x1125e1f thru 0x112c5a5 (0x6787 bytes) 2024-09-21T22:50:13.856 INFO:teuthology.orchestra.run.smithi094.stdout:239 trunc from 0x99409ca to 0x232e26 2024-09-21T22:50:13.861 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:13.849+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:13.945 INFO:teuthology.orchestra.run.smithi094.stdout:240 write 0xcb3f1db thru 0xcb43149 (0x3f6f bytes) 2024-09-21T22:50:13.954 INFO:teuthology.orchestra.run.smithi094.stdout:241 writesame 0x35094 thru 0x3d0a9 data_size 0xf(0x8016 bytes) 2024-09-21T22:50:13.955 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:13.950+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:13.989 INFO:teuthology.orchestra.run.smithi094.stdout:242 trunc from 0xcb4314a to 0xfc634d 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1768 3.1d e29/24 3:bb619233:::rbd_object_map.111ea6ec99f1:head v 29'122, mlcod=29'120)", 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.992452+0000", 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001886286, 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019337689999999999, 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992452+0000", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992452+0000", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992454+0000", 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9539999999999998e-06 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992466+0000", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992496+0000", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9295999999999999e-05 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992509+0000", 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3617e-05 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992672+0000", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016304199999999999 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.992738+0000", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6045999999999998e-05 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.994377+0000", 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016381639999999999 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:50:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-21T22:50:13.998 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:14.010 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:14.014 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.039 INFO:teuthology.orchestra.run.smithi094.stdout:245 write 0x6233cf2 thru 0x624306e (0xf37d bytes) 2024-09-21T22:50:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:246 writesame 0x268a376 thru 0x269765a data_size 0x1(0xd2e5 bytes) 2024-09-21T22:50:14.055 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:14.049+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:14.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:50:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:247 read 0x984a63 thru 0x99180f (0xcdad bytes) 2024-09-21T22:50:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:249 trunc from 0x624306f to 0x4e61844 2024-09-21T22:50:14.137 INFO:teuthology.orchestra.run.smithi094.stdout:250 writesame 0xa31cb70 thru 0xa31f396 data_size 0x1(0x2827 bytes) 2024-09-21T22:50:14.138 INFO:teuthology.orchestra.run.smithi094.stdout:251 read 0x2fccb55 thru 0x2fcd3a5 (0x851 bytes) 2024-09-21T22:50:14.148 INFO:teuthology.orchestra.run.smithi094.stdout:252 write 0xb4add77 thru 0xb4ba428 (0xc6b2 bytes) 2024-09-21T22:50:14.156 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:14.150+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:14.160 INFO:teuthology.orchestra.run.smithi094.stdout:253 trunc from 0xb4ba429 to 0xaf655c9 2024-09-21T22:50:14.174 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:14.174 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:14.175 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:14.175 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:14.175 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:14.179 INFO:teuthology.orchestra.run.smithi094.stdout:254 read 0x112840d thru 0x1128832 (0x426 bytes) 2024-09-21T22:50:14.179 INFO:teuthology.orchestra.run.smithi094.stdout:256 read 0xa64305 thru 0xa6e366 (0xa062 bytes) 2024-09-21T22:50:14.179 INFO:teuthology.orchestra.run.smithi094.stdout:257 read 0x4119871 thru 0x4124830 (0xafc0 bytes) 2024-09-21T22:50:14.179 INFO:teuthology.orchestra.run.smithi094.stdout:258 write 0x68e1146 thru 0x68eb993 (0xa84e bytes) 2024-09-21T22:50:14.190 INFO:teuthology.orchestra.run.smithi094.stdout:260 punch from 0x301b1b5 to 0x301ba1f, (0x86a bytes) 2024-09-21T22:50:14.190 INFO:teuthology.orchestra.run.smithi094.stdout:261 read 0x876a993 thru 0x87762eb (0xb959 bytes) 2024-09-21T22:50:14.190 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.190 INFO:teuthology.orchestra.run.smithi094.stdout:263 write 0x8f11867 thru 0x8f18f40 (0x76da bytes) 2024-09-21T22:50:14.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:14.200 INFO:teuthology.orchestra.run.smithi094.stdout:264 punch from 0x24133ad to 0x2415021, (0x1c74 bytes) 2024-09-21T22:50:14.200 INFO:teuthology.orchestra.run.smithi094.stdout:265 punch from 0x95d7059 to 0x95e3e34, (0xcddb bytes) 2024-09-21T22:50:14.200 INFO:teuthology.orchestra.run.smithi094.stdout:266 read 0x43441de thru 0x434ed87 (0xabaa bytes) 2024-09-21T22:50:14.200 INFO:teuthology.orchestra.run.smithi094.stdout:268 write 0x2eeeab thru 0x2fc9b6 (0xdb0c bytes) 2024-09-21T22:50:14.211 INFO:teuthology.orchestra.run.smithi094.stdout:270 write 0x457d590 thru 0x457e249 (0xcba bytes) 2024-09-21T22:50:14.214 INFO:teuthology.orchestra.run.smithi094.stdout:272 read 0x1c56358 thru 0x1c5f21a (0x8ec3 bytes) 2024-09-21T22:50:14.214 INFO:teuthology.orchestra.run.smithi094.stdout:273 read 0x500cc3a thru 0x50108d1 (0x3c98 bytes) 2024-09-21T22:50:14.219 INFO:teuthology.orchestra.run.smithi094.stdout:274 trunc from 0xaf655c9 to 0x7e2517f 2024-09-21T22:50:14.231 INFO:teuthology.orchestra.run.smithi094.stdout:275 read 0x1e8a84 thru 0x1efcd8 (0x7255 bytes) 2024-09-21T22:50:14.232 INFO:teuthology.orchestra.run.smithi094.stdout:276 punch from 0x57b4185 to 0x57c2a7e, (0xe8f9 bytes) 2024-09-21T22:50:14.234 INFO:teuthology.orchestra.run.smithi094.stdout:277 trunc from 0x7e2517f to 0x98337bd 2024-09-21T22:50:14.241 INFO:teuthology.orchestra.run.smithi094.stdout:279 punch from 0x7581668 to 0x758a43b, (0x8dd3 bytes) 2024-09-21T22:50:14.241 INFO:teuthology.orchestra.run.smithi094.stdout:281 read 0x84786f3 thru 0x8485e47 (0xd755 bytes) 2024-09-21T22:50:14.241 INFO:teuthology.orchestra.run.smithi094.stdout:283 punch from 0x8e15b9a to 0x8e1fe4d, (0xa2b3 bytes) 2024-09-21T22:50:14.242 INFO:teuthology.orchestra.run.smithi094.stdout:284 read 0x3c82987 thru 0x3c89298 (0x6912 bytes) 2024-09-21T22:50:14.253 INFO:teuthology.orchestra.run.smithi094.stdout:285 write 0xce81bb5 thru 0xce835bc (0x1a08 bytes) 2024-09-21T22:50:14.256 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:14.253+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:14.257 INFO:teuthology.orchestra.run.smithi094.stdout:286 write 0xccb527a thru 0xccbe9e5 (0x976c bytes) 2024-09-21T22:50:14.280 INFO:teuthology.orchestra.run.smithi094.stdout:287 trunc from 0xce835bd to 0x13ae2ce 2024-09-21T22:50:14.300 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.326 INFO:teuthology.orchestra.run.smithi094.stdout:289 writesame 0xc84b9e2 thru 0xc85a367 data_size 0x47(0xe986 bytes) 2024-09-21T22:50:14.326 INFO:teuthology.orchestra.run.smithi094.stdout:290 write 0xa28097a thru 0xa282c1f (0x22a6 bytes) 2024-09-21T22:50:14.330 INFO:teuthology.orchestra.run.smithi094.stdout:293 read 0xaa1c240 thru 0xaa24ec7 (0x8c88 bytes) 2024-09-21T22:50:14.340 INFO:teuthology.orchestra.run.smithi094.stdout:294 write 0xcc29eb5 thru 0xcc2a407 (0x553 bytes) 2024-09-21T22:50:14.347 INFO:teuthology.orchestra.run.smithi094.stdout:295 write 0x3ed608a thru 0x3ee15a6 (0xb51d bytes) 2024-09-21T22:50:14.357 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:14.353+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:14.358 INFO:teuthology.orchestra.run.smithi094.stdout:297 read 0x7bb73af thru 0x7bbb33b (0x3f8d bytes) 2024-09-21T22:50:14.358 INFO:teuthology.orchestra.run.smithi094.stdout:298 read 0x5b19d31 thru 0x5b1e9bd (0x4c8d bytes) 2024-09-21T22:50:14.359 INFO:teuthology.orchestra.run.smithi094.stdout:301 trunc from 0xcc2a408 to 0xbe8bab2 2024-09-21T22:50:14.370 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:451 3.1a 3:58b09410:::rbd_data.111ecbf88771.0000000000000072:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.8993193509999999, 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074162789999999996, 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 451 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469994+0000", 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.626e-06 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470004+0000", 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4159999999999993e-06 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470020+0000", 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6447e-05 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470031+0000", 2024-09-21T22:50:14.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470093+0000", 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2024000000000001e-05 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470206+0000", 2024-09-21T22:50:14.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113238 2024-09-21T22:50:14.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:14.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.471324+0000", 2024-09-21T22:50:14.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001117364 2024-09-21T22:50:14.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477262+0000", 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059383550000000002 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477344+0000", 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1831999999999998e-05 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477378+0000", 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3618000000000002e-05 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477409+0000", 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1263000000000002e-05 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:452 3.a 3:56d9bfad:::rbd_data.111ecbf88771.0000000000000073:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.899205327, 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072291539999999998, 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 452 2024-09-21T22:50:14.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470108+0000", 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.08e-06 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470114+0000", 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470124+0000", 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0465999999999999e-05 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470132+0000", 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0670000000000003e-06 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470202+0000", 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0653000000000003e-05 2024-09-21T22:50:14.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470412+0000", 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209817 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477222+0000", 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068098330000000004 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477263+0000", 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0694e-05 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477336+0000", 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3250999999999994e-05 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:590 3.14 3:2d3a55cb:::rbd_data.111ecbf88771.000000000000008a:head [truncate 335872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.641199+0000", 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.728112718, 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053890049999999997, 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 590 2024-09-21T22:50:14.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641199+0000", 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641199+0000", 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641201+0000", 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1239999999999999e-06 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641212+0000", 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.058e-05 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641237+0000", 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5335999999999999e-05 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641255+0000", 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7467000000000001e-05 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641419+0000", 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016457200000000001 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641630+0000", 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021088199999999999 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.641967+0000", 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033720000000000001 2024-09-21T22:50:14.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.646189+0000", 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004221898 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.646379+0000", 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189641 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.646519+0000", 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014022899999999999 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.646588+0000", 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9076000000000001e-05 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:627 3.a 3:52ce517f:::rbd_data.111ecbf88771.0000000000000131:head [set-alloc-hint object_size 524288 write_size 524288,write 163840~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.687898+0000", 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.681413708, 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058689079999999999, 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 627 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687898+0000", 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687898+0000", 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687900+0000", 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7e-06 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687927+0000", 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6696999999999999e-05 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687954+0000", 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7209999999999999e-05 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.687971+0000", 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7223000000000001e-05 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.689618+0000", 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001646701 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.690045+0000", 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042747299999999999 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.693462+0000", 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034166169999999998 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.693556+0000", 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4449000000000003e-05 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.693767+0000", 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210838 2024-09-21T22:50:14.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:653 3.a 3:551485c8:::rbd_data.111ecbf88771.00000000000000fa:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.6476324789999999, 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082106869999999995, 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 653 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721681+0000", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5689999999999999e-06 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721689+0000", 2024-09-21T22:50:14.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721709+0000", 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9343000000000001e-05 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721718+0000", 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4150000000000005e-06 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721783+0000", 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535000000000005e-05 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721876+0000", 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3410000000000002e-05 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729625+0000", 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077490689999999999 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729751+0000", 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126062 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729890+0000", 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013910300000000001 2024-09-21T22:50:14.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:657 3.a 3:52c103dd:::rbd_data.111ecbf88771.00000000000000fe:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.724181+0000", 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.645130948, 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059727069999999998, 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 657 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724181+0000", 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724181+0000", 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724183+0000", 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.531e-06 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724191+0000", 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6410000000000008e-06 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724214+0000", 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2572e-05 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724241+0000", 2024-09-21T22:50:14.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6979e-05 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724479+0000", 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023861499999999999 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.724852+0000", 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037301700000000001 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729910+0000", 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005057387 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729996+0000", 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6463000000000002e-05 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.730154+0000", 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015750200000000001 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:846 3.a 3:551485c8:::rbd_data.111ecbf88771.00000000000000fa:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.893306+0000", 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4760057950000001, 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053095169999999997, 2024-09-21T22:50:14.411 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 846 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893306+0000", 2024-09-21T22:50:14.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893306+0000", 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893307+0000", 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:50:14.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893312+0000", 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0989999999999998e-06 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893325+0000", 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2619000000000001e-05 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893333+0000", 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6829999999999997e-06 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893479+0000", 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014563099999999999 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.893589+0000", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109719 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898291+0000", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047017140000000001 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898421+0000", 2024-09-21T22:50:14.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013094499999999999 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898616+0000", 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019419000000000001 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:850 3.a 3:52c103dd:::rbd_data.111ecbf88771.00000000000000fe:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.895831+0000", 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.473481158, 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054279180000000003, 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 850 2024-09-21T22:50:14.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895831+0000", 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895831+0000", 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895832+0000", 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9800000000000002e-07 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895836+0000", 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5310000000000003e-06 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895848+0000", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1814000000000001e-05 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895857+0000", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7490000000000004e-06 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895920+0000", 2024-09-21T22:50:14.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2868999999999997e-05 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.895984+0000", 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4547000000000004e-05 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901129+0000", 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051448459999999998 2024-09-21T22:50:14.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901182+0000", 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2697e-05 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901259+0000", 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6966999999999997e-05 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:854 3.1a 3:589c7a26:::rbd_data.111ecbf88771.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.898347+0000", 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4709651479999999, 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060357479999999996, 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 854 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898347+0000", 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898347+0000", 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898348+0000", 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898362+0000", 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3896999999999999e-05 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898377+0000", 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5053e-05 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898388+0000", 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0767999999999999e-05 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898477+0000", 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9655999999999997e-05 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898556+0000", 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8557e-05 2024-09-21T22:50:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.898664+0000", 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108484 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899588+0000", 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000923317 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904283+0000", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004695584 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904340+0000", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.626e-05 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904383+0000", 2024-09-21T22:50:14.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2976000000000003e-05 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:856 3.1d e28/24 3:bb3b272c:::rbd_data.111ecbf88771.0000000000000176:head v 28'54, mlcod=28'52)", 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.899025+0000", 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.47028713, 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054847239999999998, 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899025+0000", 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899025+0000", 2024-09-21T22:50:14.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899026+0000", 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.674e-06 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899038+0000", 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1613999999999999e-05 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899061+0000", 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3354e-05 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899082+0000", 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0191999999999999e-05 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899159+0000", 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6865000000000004e-05 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.899172+0000", 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3730999999999999e-05 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904449+0000", 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052765149999999999 2024-09-21T22:50:14.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904510+0000", 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0779000000000002e-05 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:864 3.1a 3:5922ac48:::rbd_data.111ecbf88771.00000000000001c6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.901582+0000", 2024-09-21T22:50:14.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467729504, 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056093870000000004, 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 864 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.437 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901582+0000", 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901582+0000", 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901583+0000", 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3099999999999996e-07 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901588+0000", 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6399999999999996e-06 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901600+0000", 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2027e-05 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901607+0000", 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.43e-06 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901669+0000", 2024-09-21T22:50:14.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1785999999999994e-05 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901718+0000", 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8856000000000001e-05 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.901810+0000", 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1553999999999998e-05 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.902659+0000", 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00084935399999999997 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.906984+0000", 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043247870000000001 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.907097+0000", 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112921 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.907192+0000", 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5100999999999996e-05 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:866 3.a 3:50c41eb6:::rbd_data.111ecbf88771.00000000000001d5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.904210+0000", 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4651018740000001, 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061801490000000002, 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 866 2024-09-21T22:50:14.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904210+0000", 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904210+0000", 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904217+0000", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9040000000000003e-06 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904225+0000", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.14e-06 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904249+0000", 2024-09-21T22:50:14.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4107000000000001e-05 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904265+0000", 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5943e-05 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904409+0000", 2024-09-21T22:50:14.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014333299999999999 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.904493+0000", 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4351000000000006e-05 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.909860+0000", 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053672049999999999 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.909951+0000", 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1382e-05 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.910390+0000", 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043878400000000001 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1008 3.14 3:2c7bff05:::rbd_data.111ecbf88771.0000000000000030:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2103813400000001, 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068041639999999997, 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1008 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158933+0000", 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158945+0000", 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158980+0000", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4980000000000001e-05 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159000+0000", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0015999999999999e-05 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159308+0000", 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030743099999999999 2024-09-21T22:50:14.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159539+0000", 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023104500000000001 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.160019+0000", 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048036299999999998 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.162010+0000", 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019912760000000002 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165428+0000", 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034175820000000002 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165584+0000", 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015635900000000001 2024-09-21T22:50:14.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165735+0000", 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015054900000000001 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1033 3.1b e29/24 3:dd3d94e9:::rbd_data.111ecbf88771.0000000000000074:head v 29'9, mlcod=28'6)", 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.195906981, 2024-09-21T22:50:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070721250000000003, 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173407+0000", 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6759999999999999e-06 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173420+0000", 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3224000000000001e-05 2024-09-21T22:50:14.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173446+0000", 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6118999999999998e-05 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173465+0000", 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8542999999999998e-05 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173677+0000", 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000212936 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173731+0000", 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3981999999999999e-05 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180381+0000", 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066499920000000004 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180477+0000", 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5653000000000001e-05 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1347 3.a 3:57948078:::rbd_data.111ea6ec99f1.0000000000000061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.834285+0000", 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.53502740800000004, 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0065140290000000002, 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1347 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834285+0000", 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834285+0000", 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834286+0000", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7300000000000004e-07 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834292+0000", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8530000000000003e-06 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834305+0000", 2024-09-21T22:50:14.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2184000000000001e-05 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834315+0000", 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0019e-05 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834452+0000", 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013712899999999999 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834669+0000", 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021687900000000001 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.840596+0000", 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059278300000000003 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.840668+0000", 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2033000000000006e-05 2024-09-21T22:50:14.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.840799+0000", 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013012899999999999 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1348 3.14 3:28e134e3:::rbd_data.111ea6ec99f1.0000000000000062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.834336+0000", 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.53497547599999995, 2024-09-21T22:50:14.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056025370000000003, 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1348 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834336+0000", 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834336+0000", 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834337+0000", 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8400000000000002e-07 2024-09-21T22:50:14.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834343+0000", 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4029999999999998e-06 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834361+0000", 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7727000000000002e-05 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834374+0000", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3390000000000001e-05 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834488+0000", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114109 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.834696+0000", 2024-09-21T22:50:14.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020814100000000001 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.835217+0000", 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00052116 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.839282+0000", 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040646959999999996 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.839707+0000", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042516599999999999 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.839837+0000", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012970399999999999 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.839939+0000", 2024-09-21T22:50:14.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102057 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1360 3.1a 3:59a803a6:::rbd_data.111ea6ec99f1.0000000000000150:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.841222+0000", 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.528089959, 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006308485, 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1360 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841222+0000", 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841222+0000", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841224+0000", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0669999999999999e-06 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841235+0000", 2024-09-21T22:50:14.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0956e-05 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841268+0000", 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2626999999999998e-05 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841287+0000", 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9565999999999999e-05 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841540+0000", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025230499999999998 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841813+0000", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027367400000000001 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.842009+0000", 2024-09-21T22:50:14.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000196126 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.844149+0000", 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021392339999999998 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.847368+0000", 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032194569999999998 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.847466+0000", 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7676999999999993e-05 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.847530+0000", 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4795999999999994e-05 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1361 3.13 e29/24 3:c981ce01:::rbd_data.111ea6ec99f1.0000000000000151:head v 29'19, mlcod=29'17)", 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.841955+0000", 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52735696099999996, 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055311199999999996, 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841955+0000", 2024-09-21T22:50:14.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841955+0000", 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841957+0000", 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6959999999999999e-06 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841970+0000", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3567000000000001e-05 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.841994+0000", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.334e-05 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.842031+0000", 2024-09-21T22:50:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7656e-05 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.842131+0000", 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9779999999999997e-05 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.842202+0000", 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1014999999999997e-05 2024-09-21T22:50:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.847376+0000", 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051743170000000003 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.847486+0000", 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109749 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1723 3.1d e29/24 3:bb619233:::rbd_object_map.111ea6ec99f1:head v 29'107, mlcod=29'103)", 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.875446+0000", 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.493865895, 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053568959999999999, 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875446+0000", 2024-09-21T22:50:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875446+0000", 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875447+0000", 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3099999999999996e-07 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875459+0000", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2071e-05 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875476+0000", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875488+0000", 2024-09-21T22:50:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1866e-05 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875581+0000", 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3101000000000002e-05 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.875643+0000", 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2424999999999999e-05 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.880734+0000", 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050912600000000002 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.880803+0000", 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8578999999999995e-05 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1943 3.14 3:2c0dfb14:::rbd_data.111ea6ec99f1.000000000000033e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.286673+0000", 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.082638748999999997, 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055906979999999998, 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1943 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286673+0000", 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286673+0000", 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286675+0000", 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.474e-06 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286680+0000", 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286692+0000", 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1715e-05 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286698+0000", 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286751+0000", 2024-09-21T22:50:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2638e-05 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286806+0000", 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4818000000000003e-05 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.286905+0000", 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9874e-05 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.291985+0000", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050796779999999998 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.292198+0000", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021326400000000001 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.292229+0000", 2024-09-21T22:50:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0664999999999999e-05 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.292264+0000", 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4867999999999999e-05 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.497 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:14.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:14.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:302 read 0x6ce8a2a thru 0x6cf2696 (0x9c6d bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:305 write 0xe02b678 thru 0xe031db2 (0x673b bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:306 write 0x6097348 thru 0x60a4e65 (0xdb1e bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:310 writesame 0xcde333a thru 0xcde86af data_size 0x9(0x5376 bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:311 write 0xa310ab1 thru 0xa31330e (0x285e bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:313 write 0x1f61776 thru 0x1f63909 (0x2194 bytes) 2024-09-21T22:50:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.502 INFO:teuthology.orchestra.run.smithi094.stdout:315 write 0xaf870ff thru 0xaf8c060 (0x4f62 bytes) 2024-09-21T22:50:14.502 INFO:teuthology.orchestra.run.smithi094.stdout:317 read 0xbaa6342 thru 0xbaaa6e3 (0x43a2 bytes) 2024-09-21T22:50:14.502 INFO:teuthology.orchestra.run.smithi094.stdout:319 write 0xeba49bb thru 0xebb26cb (0xdd11 bytes) 2024-09-21T22:50:14.502 INFO:teuthology.orchestra.run.smithi094.stdout:321 read 0xeb9dc7d thru 0xeba192b (0x3caf bytes) 2024-09-21T22:50:14.502 INFO:teuthology.orchestra.run.smithi094.stdout:322 trunc from 0xebb26cc to 0x5191656 2024-09-21T22:50:14.502 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:14.493+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:323 read 0x2ace0f2 thru 0x2add37c (0xf28b bytes) 2024-09-21T22:50:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:326 read 0x41882d7 thru 0x4188e65 (0xb8f bytes) 2024-09-21T22:50:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:327 punch from 0x376e2af to 0x3770937, (0x2688 bytes) 2024-09-21T22:50:14.533 INFO:teuthology.orchestra.run.smithi094.stdout:328 writesame 0xa3093fb thru 0xa31130f data_size 0x1(0x7f15 bytes) 2024-09-21T22:50:14.552 INFO:teuthology.orchestra.run.smithi094.stdout:329 write 0xebab152 thru 0xebb1ef1 (0x6da0 bytes) 2024-09-21T22:50:14.559 INFO:teuthology.orchestra.run.smithi094.stdout:330 punch from 0x3407597 to 0x34101a0, (0x8c09 bytes) 2024-09-21T22:50:14.559 INFO:teuthology.orchestra.run.smithi094.stdout:331 write 0xbe27fb5 thru 0xbe33517 (0xb563 bytes) 2024-09-21T22:50:14.566 INFO:teuthology.orchestra.run.smithi094.stdout:332 write 0xa78e911 thru 0xa797aab (0x919b bytes) 2024-09-21T22:50:14.573 INFO:teuthology.orchestra.run.smithi094.stdout:333 read 0x26f0fda thru 0x26f4b03 (0x3b2a bytes) 2024-09-21T22:50:14.573 INFO:teuthology.orchestra.run.smithi094.stdout:334 write 0xa7a1e0e thru 0xa7afdb9 (0xdfac bytes) 2024-09-21T22:50:14.582 INFO:teuthology.orchestra.run.smithi094.stdout:337 read 0xa7507ff thru 0xa752690 (0x1e92 bytes) 2024-09-21T22:50:14.582 INFO:teuthology.orchestra.run.smithi094.stdout:338 read 0xb72e9db thru 0xb7388c3 (0x9ee9 bytes) 2024-09-21T22:50:14.593 INFO:teuthology.orchestra.run.smithi094.stdout:339 trunc from 0xebb1ef2 to 0x768d70b 2024-09-21T22:50:14.600 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:14.594+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:14.615 INFO:teuthology.orchestra.run.smithi094.stdout:340 write 0x4d91084 thru 0x4d9e914 (0xd891 bytes) 2024-09-21T22:50:14.627 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.627 INFO:teuthology.orchestra.run.smithi094.stdout:348 writesame 0x35eceb2 thru 0x35f32c8 data_size 0x49(0x6417 bytes) 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2166 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.628627+0000", 2024-09-21T22:50:14.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0042903170000000001, 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043249389999999999, 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2166 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:14.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628627+0000", 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628627+0000", 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628628+0000", 2024-09-21T22:50:14.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3e-07 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628632+0000", 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628642+0000", 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.268e-06 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.628650+0000", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0090000000000007e-06 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.629398+0000", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074811799999999998 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.629523+0000", 2024-09-21T22:50:14.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012578099999999999 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.629844+0000", 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032067100000000002 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-21T22:50:14.637 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:14.638 INFO:teuthology.orchestra.run.smithi094.stdout:351 punch from 0x53e236d to 0x53e6485, (0x4118 bytes) 2024-09-21T22:50:14.638 INFO:teuthology.orchestra.run.smithi094.stdout:352 write 0x72dab9c thru 0x72e9947 (0xedac bytes) 2024-09-21T22:50:14.644 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.644 INFO:teuthology.orchestra.run.smithi094.stdout:356 punch from 0x471fa38 to 0x472d4b6, (0xda7e bytes) 2024-09-21T22:50:14.644 INFO:teuthology.orchestra.run.smithi094.stdout:357 read 0x4e0ead3 thru 0x4e1d0d5 (0xe603 bytes) 2024-09-21T22:50:14.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:14.654 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:14.659 INFO:teuthology.orchestra.run.smithi094.stdout:358 write 0xcd2bea0 thru 0xcd3235f (0x64c0 bytes) 2024-09-21T22:50:14.667 INFO:teuthology.orchestra.run.smithi094.stdout:359 punch from 0x5104bd to 0x513ca9, (0x37ec bytes) 2024-09-21T22:50:14.667 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:361 read 0xc742827 thru 0xc751934 (0xf10e bytes) 2024-09-21T22:50:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:363 write 0x4c789e5 thru 0x4c8333d (0xa959 bytes) 2024-09-21T22:50:14.671 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:14.677 INFO:teuthology.orchestra.run.smithi094.stdout:365 read 0x72e5147 thru 0x72eba6d (0x6927 bytes) 2024-09-21T22:50:14.678 INFO:teuthology.orchestra.run.smithi094.stdout:366 writesame 0x43154ed thru 0x43213df data_size 0x1(0xbef3 bytes) 2024-09-21T22:50:14.689 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:14.689 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:14.696 INFO:teuthology.orchestra.run.smithi094.stdout:369 punch from 0x1cb0bf3 to 0x1cb901b, (0x8428 bytes) 2024-09-21T22:50:14.697 INFO:teuthology.orchestra.run.smithi094.stdout:370 write 0x3f27070 thru 0x3f30ed7 (0x9e68 bytes) 2024-09-21T22:50:14.702 INFO:teuthology.orchestra.run.smithi094.stdout:371 writesame 0xa6337c8 thru 0xa643470 data_size 0x1(0xfca9 bytes) 2024-09-21T22:50:14.707 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:14.707 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:14.720 INFO:teuthology.orchestra.run.smithi094.stdout:373 writesame 0x8649174 thru 0x8650c67 data_size 0x81(0x7af4 bytes) 2024-09-21T22:50:14.727 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:14.728 INFO:teuthology.orchestra.run.smithi094.stdout:375 read 0x7983906 thru 0x798c06d (0x8768 bytes) 2024-09-21T22:50:14.728 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.728 INFO:teuthology.orchestra.run.smithi094.stdout:377 writesame 0x1f08aef thru 0x1f130e6 data_size 0x71(0xa5f8 bytes) 2024-09-21T22:50:14.735 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.735 INFO:teuthology.orchestra.run.smithi094.stdout:380 read 0xaf53a4c thru 0xaf5cc33 (0x91e8 bytes) 2024-09-21T22:50:14.735 INFO:teuthology.orchestra.run.smithi094.stdout:382 read 0x66fda5 thru 0x6717ed (0x1a49 bytes) 2024-09-21T22:50:14.736 INFO:teuthology.orchestra.run.smithi094.stdout:383 write 0xbd234cd thru 0xbd2d1ab (0x9cdf bytes) 2024-09-21T22:50:14.736 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:14.734+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:14.745 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.747 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:14.747 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:14.747 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:14.754 INFO:teuthology.orchestra.run.smithi094.stdout:385 writesame 0xd82912e thru 0xd8383d2 data_size 0x1(0xf2a5 bytes) 2024-09-21T22:50:14.754 INFO:teuthology.orchestra.run.smithi094.stdout:386 write 0x73e9f55 thru 0x73f6ada (0xcb86 bytes) 2024-09-21T22:50:14.767 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:14.770 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:14.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:14.781 INFO:teuthology.orchestra.run.smithi094.stdout:388 trunc from 0xd8383d3 to 0xa8a4d3 2024-09-21T22:50:14.801 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:14.798+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:14.805 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.805 INFO:teuthology.orchestra.run.smithi094.stdout:390 punch from 0x9d5fd8 to 0x9d8634, (0x265c bytes) 2024-09-21T22:50:14.805 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.805 INFO:teuthology.orchestra.run.smithi094.stdout:394 punch from 0x2f7e07 to 0x2fc245, (0x443e bytes) 2024-09-21T22:50:14.805 INFO:teuthology.orchestra.run.smithi094.stdout:395 read 0x396f5b thru 0x3a135c (0xa402 bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:396 writesame 0x3768d1c thru 0x376f06f data_size 0x4e(0x6354 bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:399 punch from 0x25f2824 to 0x260101a, (0xe7f6 bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:400 read 0x1f69659 thru 0x1f6bbd1 (0x2579 bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:401 read 0xc9c414 thru 0xca90c1 (0xccae bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:402 read 0x14b413c thru 0x14bedd6 (0xac9b bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:404 punch from 0x1528b16 to 0x152bf38, (0x3422 bytes) 2024-09-21T22:50:14.817 INFO:teuthology.orchestra.run.smithi094.stdout:405 read 0x278d065 thru 0x27961f3 (0x918f bytes) 2024-09-21T22:50:14.826 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:14.826 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:14.836 INFO:teuthology.orchestra.run.smithi094.stdout:406 writesame 0xb1309bc thru 0xb139f5e data_size 0x3(0x95a3 bytes) 2024-09-21T22:50:14.836 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.837 INFO:teuthology.orchestra.run.smithi094.stdout:408 punch from 0x6d390ba to 0x6d3afcb, (0x1f11 bytes) 2024-09-21T22:50:14.837 INFO:teuthology.orchestra.run.smithi094.stdout:410 write 0x13701db thru 0x1375b6e (0x5994 bytes) 2024-09-21T22:50:14.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:14.845 INFO:teuthology.orchestra.run.smithi094.stdout:412 write 0x8c755f8 thru 0x8c7ed6b (0x9774 bytes) 2024-09-21T22:50:14.856 INFO:teuthology.orchestra.run.smithi094.stdout:414 writesame 0x20e1dc thru 0x215ee3 data_size 0x4(0x7d08 bytes) 2024-09-21T22:50:14.862 INFO:teuthology.orchestra.run.smithi094.stdout:416 write 0x7045493 thru 0x705536c (0xfeda bytes) 2024-09-21T22:50:14.872 INFO:teuthology.orchestra.run.smithi094.stdout:417 punch from 0x6574e23 to 0x657eca6, (0x9e83 bytes) 2024-09-21T22:50:14.872 INFO:teuthology.orchestra.run.smithi094.stdout:418 read 0x284c5fc thru 0x2855edb (0x98e0 bytes) 2024-09-21T22:50:14.878 INFO:teuthology.orchestra.run.smithi094.stdout:419 trunc from 0xb139f5f to 0x7e92394 2024-09-21T22:50:14.901 INFO:teuthology.orchestra.run.smithi094.stdout:421 trunc from 0x7e92394 to 0xb3f080c 2024-09-21T22:50:14.902 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:14.897+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:14.910 INFO:teuthology.orchestra.run.smithi094.stdout:422 read 0x4884b22 thru 0x488522d (0x70c bytes) 2024-09-21T22:50:14.914 INFO:teuthology.orchestra.run.smithi094.stdout:423 trunc from 0xb3f080c to 0xdb4571d 2024-09-21T22:50:14.923 INFO:teuthology.orchestra.run.smithi094.stdout:425 read 0x82fc037 thru 0x8302c36 (0x6c00 bytes) 2024-09-21T22:50:14.923 INFO:teuthology.orchestra.run.smithi094.stdout:426 read 0x5cbccd6 thru 0x5cc309c (0x63c7 bytes) 2024-09-21T22:50:14.924 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.924 INFO:teuthology.orchestra.run.smithi094.stdout:430 punch from 0x76a34d0 to 0x76a9fbc, (0x6aec bytes) 2024-09-21T22:50:14.924 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:14.924 INFO:teuthology.orchestra.run.smithi094.stdout:433 writesame 0x4595e5c thru 0x45a27e0 data_size 0x1(0xc985 bytes) 2024-09-21T22:50:14.943 INFO:teuthology.orchestra.run.smithi094.stdout:435 punch from 0x21b3e5a to 0x21bf4a8, (0xb64e bytes) 2024-09-21T22:50:14.949 INFO:teuthology.orchestra.run.smithi094.stdout:436 trunc from 0xdb4571d to 0xa105fb4 2024-09-21T22:50:14.959 INFO:teuthology.orchestra.run.smithi094.stdout:437 write 0x2a038de thru 0x2a0c74b (0x8e6e bytes) 2024-09-21T22:50:14.980 INFO:teuthology.orchestra.run.smithi094.stdout:439 write 0xbddb540 thru 0xbde707b (0xbb3c bytes) 2024-09-21T22:50:14.989 INFO:teuthology.orchestra.run.smithi094.stdout:440 punch from 0x8e8e694 to 0x8e949e0, (0x634c bytes) 2024-09-21T22:50:14.989 INFO:teuthology.orchestra.run.smithi094.stdout:441 read 0x9a69a7c thru 0x9a78364 (0xe8e9 bytes) 2024-09-21T22:50:15.003 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:14.997+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:15.004 INFO:teuthology.orchestra.run.smithi094.stdout:442 trunc from 0xbde707c to 0x5e0a31c 2024-09-21T22:50:15.013 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.014 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.024 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.028 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:448 writesame 0x7adc647 thru 0x7ae0a0d data_size 0x1(0x43c7 bytes) 2024-09-21T22:50:15.046 INFO:teuthology.orchestra.run.smithi094.stdout:449 trunc from 0x7ae0a0e to 0x53bc854 2024-09-21T22:50:15.056 INFO:teuthology.orchestra.run.smithi094.stdout:450 punch from 0x2b61740 to 0x2b6d62e, (0xbeee bytes) 2024-09-21T22:50:15.073 INFO:teuthology.orchestra.run.smithi094.stdout:451 write 0x891e8fe thru 0x8925655 (0x6d58 bytes) 2024-09-21T22:50:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:50:15.087 INFO:teuthology.orchestra.run.smithi094.stdout:452 write 0x1dcde3 thru 0x1ddff9 (0x1217 bytes) 2024-09-21T22:50:15.089 INFO:teuthology.orchestra.run.smithi094.stdout:453 punch from 0x7186082 to 0x7195596, (0xf514 bytes) 2024-09-21T22:50:15.090 INFO:teuthology.orchestra.run.smithi094.stdout:454 punch from 0x374351c to 0x37465a6, (0x308a bytes) 2024-09-21T22:50:15.090 INFO:teuthology.orchestra.run.smithi094.stdout:455 write 0x85725b1 thru 0x85805e1 (0xe031 bytes) 2024-09-21T22:50:15.103 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:15.098+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:15.116 INFO:teuthology.orchestra.run.smithi094.stdout:456 trunc from 0x8925656 to 0x48fc75 2024-09-21T22:50:15.161 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.161 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:15.176 INFO:teuthology.orchestra.run.smithi094.stdout:464 write 0x80e52f9 thru 0x80e6a95 (0x179d bytes) 2024-09-21T22:50:15.184 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.185 INFO:teuthology.orchestra.run.smithi094.stdout:468 read 0x23437ae thru 0x23438ef (0x142 bytes) 2024-09-21T22:50:15.193 INFO:teuthology.orchestra.run.smithi094.stdout:469 trunc from 0x80e6a96 to 0x36aee0b 2024-09-21T22:50:15.213 INFO:teuthology.orchestra.run.smithi094.stdout:470 write 0x6bfec9 thru 0x6cf92d (0xfa65 bytes) 2024-09-21T22:50:15.217 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:15.214+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:15.238 INFO:teuthology.orchestra.run.smithi094.stdout:473 writesame 0x5ea1eb9 thru 0x5eb058a data_size 0x13(0xe6d2 bytes) 2024-09-21T22:50:15.238 INFO:teuthology.orchestra.run.smithi094.stdout:475 read 0x36086a1 thru 0x3612dea (0xa74a bytes) 2024-09-21T22:50:15.253 INFO:teuthology.orchestra.run.smithi094.stdout:476 trunc from 0x5eb058b to 0xd2065f3 2024-09-21T22:50:15.263 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.263 INFO:teuthology.orchestra.run.smithi094.stdout:478 read 0x5dee26c thru 0x5def208 (0xf9d bytes) 2024-09-21T22:50:15.263 INFO:teuthology.orchestra.run.smithi094.stdout:479 write 0x73fd66f thru 0x740a90c (0xd29e bytes) 2024-09-21T22:50:15.276 INFO:teuthology.orchestra.run.smithi094.stdout:480 write 0x667e3e3 thru 0x6688ff2 (0xac10 bytes) 2024-09-21T22:50:15.285 INFO:teuthology.orchestra.run.smithi099.stderr:Exporting 2.7 info 2.7( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-21T22:50:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:481 read 0xcbca028 thru 0xcbccab8 (0x2a91 bytes) 2024-09-21T22:50:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:482 read 0x9f88a04 thru 0x9f91415 (0x8a12 bytes) 2024-09-21T22:50:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:484 read 0x54d3475 thru 0x54da371 (0x6efd bytes) 2024-09-21T22:50:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:486 punch from 0x4f0e2af to 0x4f1d277, (0xefc8 bytes) 2024-09-21T22:50:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:487 read 0x8d2e1bc thru 0x8d3da7b (0xf8c0 bytes) 2024-09-21T22:50:15.289 INFO:teuthology.orchestra.run.smithi099.stderr:Export successful 2024-09-21T22:50:15.302 INFO:teuthology.orchestra.run.smithi094.stdout:488 trunc from 0xd2065f3 to 0x5f8e6f2 2024-09-21T22:50:15.305 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:15.302+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:15.320 INFO:teuthology.orchestra.run.smithi094.stdout:489 write 0x284e0e thru 0x28f7b9 (0xa9ac bytes) 2024-09-21T22:50:15.326 INFO:teuthology.orchestra.run.smithi094.stdout:490 writesame 0x271f5a8 thru 0x2724301 data_size 0x1(0x4d5a bytes) 2024-09-21T22:50:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:15.376 INFO:teuthology.orchestra.run.smithi094.stdout:491 write 0xba078c3 thru 0xba155c1 (0xdcff bytes) 2024-09-21T22:50:15.387 INFO:teuthology.orchestra.run.smithi094.stdout:492 read 0xb2c6b74 thru 0xb2d22e3 (0xb770 bytes) 2024-09-21T22:50:15.387 INFO:teuthology.orchestra.run.smithi094.stdout:493 punch from 0x84adba7 to 0x84b0587, (0x29e0 bytes) 2024-09-21T22:50:15.387 INFO:teuthology.orchestra.run.smithi094.stdout:494 writesame 0x9720dd7 thru 0x97270b5 data_size 0xd(0x62df bytes) 2024-09-21T22:50:15.394 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.394 INFO:teuthology.orchestra.run.smithi094.stdout:496 punch from 0x4af5c3a to 0x4af699d, (0xd63 bytes) 2024-09-21T22:50:15.394 INFO:teuthology.orchestra.run.smithi094.stdout:497 write 0x1a3522a thru 0x1a3d9c7 (0x879e bytes) 2024-09-21T22:50:15.405 INFO:teuthology.orchestra.run.smithi094.stdout:498 punch from 0x41bb067 to 0x41c7041, (0xbfda bytes) 2024-09-21T22:50:15.406 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.406 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.406 INFO:teuthology.orchestra.run.smithi094.stdout:501 writesame 0x68c4971 thru 0x68c886b data_size 0x1(0x3efb bytes) 2024-09-21T22:50:15.409 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:15.406+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:15.430 INFO:teuthology.orchestra.run.smithi094.stdout:502 punch from 0x25032b3 to 0x25045bc, (0x1309 bytes) 2024-09-21T22:50:15.430 INFO:teuthology.orchestra.run.smithi094.stdout:503 write 0x4d5745d thru 0x4d64d29 (0xd8cd bytes) 2024-09-21T22:50:15.453 INFO:teuthology.orchestra.run.smithi094.stdout:504 trunc from 0xba155c2 to 0xe5428db 2024-09-21T22:50:15.474 INFO:teuthology.orchestra.run.smithi094.stdout:505 writesame 0xe774cb0 thru 0xe777cfd data_size 0x12(0x304e bytes) 2024-09-21T22:50:15.488 INFO:teuthology.orchestra.run.smithi094.stdout:506 trunc from 0xe777cfe to 0x91de800 2024-09-21T22:50:15.506 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:15.501+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:15.513 INFO:teuthology.orchestra.run.smithi094.stdout:507 read 0x88cba47 thru 0x88ce673 (0x2c2d bytes) 2024-09-21T22:50:15.513 INFO:teuthology.orchestra.run.smithi094.stdout:508 writesame 0x39b4fbb thru 0x39b8d9a data_size 0x16(0x3de0 bytes) 2024-09-21T22:50:15.543 INFO:teuthology.orchestra.run.smithi094.stdout:509 trunc from 0x91de800 to 0x1d8a303 2024-09-21T22:50:15.546 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.546 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:515 punch from 0x70bb42 to 0x7149bd, (0x8e7b bytes) 2024-09-21T22:50:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:516 punch from 0x5c3017 to 0x5cb3f8, (0x83e1 bytes) 2024-09-21T22:50:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:518 read 0x108182c thru 0x108957c (0x7d51 bytes) 2024-09-21T22:50:15.591 INFO:teuthology.orchestra.run.smithi094.stdout:519 write 0x6aa4fcb thru 0x6aa5034 (0x6a bytes) 2024-09-21T22:50:15.596 INFO:teuthology.orchestra.run.smithi094.stdout:520 read 0x3b12087 thru 0x3b21d88 (0xfd02 bytes) 2024-09-21T22:50:15.596 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.607 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:15.601+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:15.613 INFO:teuthology.orchestra.run.smithi094.stdout:523 trunc from 0x6aa5035 to 0xd74e719 2024-09-21T22:50:15.625 INFO:teuthology.orchestra.run.smithi094.stdout:525 write 0x3bc2f3c thru 0x3bd1b73 (0xec38 bytes) 2024-09-21T22:50:15.638 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.638 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.638 INFO:teuthology.orchestra.run.smithi094.stdout:530 write 0xc0c5f17 thru 0xc0d026e (0xa358 bytes) 2024-09-21T22:50:15.647 INFO:teuthology.orchestra.run.smithi094.stdout:532 read 0x90137e3 thru 0x901fb3a (0xc358 bytes) 2024-09-21T22:50:15.649 INFO:teuthology.orchestra.run.smithi094.stdout:533 trunc from 0xd74e719 to 0xca37692 2024-09-21T22:50:15.663 INFO:teuthology.orchestra.run.smithi094.stdout:537 write 0x295e754 thru 0x2968d5f (0xa60c bytes) 2024-09-21T22:50:15.679 INFO:teuthology.orchestra.run.smithi094.stdout:538 read 0x80ba2d9 thru 0x80bd956 (0x367e bytes) 2024-09-21T22:50:15.679 INFO:teuthology.orchestra.run.smithi094.stdout:539 write 0x69f5152 thru 0x69f52d9 (0x188 bytes) 2024-09-21T22:50:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:540 read 0xbcc628d thru 0xbcd29de (0xc752 bytes) 2024-09-21T22:50:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:541 read 0x20bb9ad thru 0x20c83ad (0xca01 bytes) 2024-09-21T22:50:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:542 write 0x4f9e13 thru 0x509989 (0xfb77 bytes) 2024-09-21T22:50:15.692 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.703 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.703 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.708 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:15.701+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:15.709 INFO:teuthology.orchestra.run.smithi094.stdout:544 trunc from 0xca37692 to 0x1a7dc8a 2024-09-21T22:50:15.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:15.782 INFO:teuthology.orchestra.run.smithi094.stdout:545 write 0xeb7c50c thru 0xeb87bcd (0xb6c2 bytes) 2024-09-21T22:50:15.794 INFO:teuthology.orchestra.run.smithi094.stdout:546 read 0x97e1c7f thru 0x97f0c5f (0xefe1 bytes) 2024-09-21T22:50:15.794 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.795 INFO:teuthology.orchestra.run.smithi094.stdout:551 read 0xdb91a9a thru 0xdb953d0 (0x3937 bytes) 2024-09-21T22:50:15.795 INFO:teuthology.orchestra.run.smithi094.stdout:553 read 0x8639c12 thru 0x86455c7 (0xb9b6 bytes) 2024-09-21T22:50:15.795 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.795 INFO:teuthology.orchestra.run.smithi094.stdout:556 write 0x78ed117 thru 0x78f9591 (0xc47b bytes) 2024-09-21T22:50:15.807 INFO:teuthology.orchestra.run.smithi094.stdout:557 punch from 0x7bd5e06 to 0x7bd5f74, (0x16e bytes) 2024-09-21T22:50:15.808 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:15.805+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:15.827 INFO:teuthology.orchestra.run.smithi094.stdout:560 trunc from 0xeb87bce to 0x26d9ef2 2024-09-21T22:50:15.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 2.7 2024-09-21T22:50:15.851 INFO:teuthology.orchestra.run.smithi094.stdout:561 read 0x1ed8c9f thru 0x1edce44 (0x41a6 bytes) 2024-09-21T22:50:15.851 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.851 INFO:teuthology.orchestra.run.smithi094.stdout:564 read 0x2006efd thru 0x2013c53 (0xcd57 bytes) 2024-09-21T22:50:15.868 INFO:teuthology.orchestra.run.smithi094.stdout:565 write 0x6e3f512 thru 0x6e4abd6 (0xb6c5 bytes) 2024-09-21T22:50:15.881 INFO:teuthology.orchestra.run.smithi094.stdout:567 read 0x3632975 thru 0x363ccfd (0xa389 bytes) 2024-09-21T22:50:15.881 INFO:teuthology.orchestra.run.smithi094.stdout:568 write 0x113c9c8 thru 0x11439a6 (0x6fdf bytes) 2024-09-21T22:50:15.887 INFO:teuthology.orchestra.run.smithi094.stdout:569 read 0x46a2fc5 thru 0x46af240 (0xc27c bytes) 2024-09-21T22:50:15.887 INFO:teuthology.orchestra.run.smithi094.stdout:570 read 0x67b0a5 thru 0x67cf4e (0x1eaa bytes) 2024-09-21T22:50:15.891 INFO:teuthology.orchestra.run.smithi094.stdout:571 trunc from 0x6e4abd7 to 0x9f9ab59 2024-09-21T22:50:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:572 punch from 0x66cca13 to 0x66d3d8f, (0x737c bytes) 2024-09-21T22:50:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:573 write 0x91c1ef4 thru 0x91d0b85 (0xec92 bytes) 2024-09-21T22:50:15.901 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:15.901 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:15.909 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:15.905+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:15.911 INFO:teuthology.orchestra.run.smithi094.stdout:574 write 0x2ab5747 thru 0x2ab7766 (0x2020 bytes) 2024-09-21T22:50:15.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:575 read 0x242f4d9 thru 0x243a06c (0xab94 bytes) 2024-09-21T22:50:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:576 write 0x850b03a thru 0x851343a (0x8401 bytes) 2024-09-21T22:50:15.925 INFO:teuthology.orchestra.run.smithi094.stdout:577 write 0x148457 thru 0x15312d (0xacd7 bytes) 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:582 read 0x3945f75 thru 0x394d4de (0x756a bytes) 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:583 punch from 0x945c5e9 to 0x9463ef5, (0x790c bytes) 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:584 read 0x6cb0bdc thru 0x6cbf4e5 (0xe90a bytes) 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:585 punch from 0x3577f56 to 0x3580d0e, (0x8db8 bytes) 2024-09-21T22:50:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:586 writesame 0x91f12db thru 0x91fe8fa data_size 0x30(0xd620 bytes) 2024-09-21T22:50:15.934 INFO:teuthology.orchestra.run.smithi094.stdout:587 punch from 0x38431d to 0x38eacd, (0xa7b0 bytes) 2024-09-21T22:50:15.934 INFO:teuthology.orchestra.run.smithi094.stdout:588 read 0x8f02116 thru 0x8f03973 (0x185e bytes) 2024-09-21T22:50:15.934 INFO:teuthology.orchestra.run.smithi094.stdout:589 read 0x1d37367 thru 0x1d3f3d3 (0x806d bytes) 2024-09-21T22:50:15.934 INFO:teuthology.orchestra.run.smithi094.stdout:591 write 0x7e71db7 thru 0x7e7a1c0 (0x840a bytes) 2024-09-21T22:50:15.942 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.942 INFO:teuthology.orchestra.run.smithi094.stdout:593 write 0x51f1cc0 thru 0x51f6dd7 (0x5118 bytes) 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:594 read 0x24a6e7e thru 0x24b3386 (0xc509 bytes) 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:596 read 0x93380a7 thru 0x933f739 (0x7693 bytes) 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:598 read 0x96e4da4 thru 0x96ee497 (0x96f4 bytes) 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:602 read 0x8989a2d thru 0x898b8ee (0x1ec2 bytes) 2024-09-21T22:50:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:603 read 0x6bcfa6 thru 0x6bdbfe (0xc59 bytes) 2024-09-21T22:50:15.949 INFO:teuthology.orchestra.run.smithi094.stdout:604 read 0x40f5039 thru 0x41047b5 (0xf77d bytes) 2024-09-21T22:50:15.949 INFO:teuthology.orchestra.run.smithi094.stdout:605 punch from 0x5dd23cb to 0x5dde361, (0xbf96 bytes) 2024-09-21T22:50:15.949 INFO:teuthology.orchestra.run.smithi094.stdout:606 read 0x31faf1 thru 0x32adf2 (0xb302 bytes) 2024-09-21T22:50:15.961 INFO:teuthology.orchestra.run.smithi094.stdout:607 write 0xb8ca621 thru 0xb8caa97 (0x477 bytes) 2024-09-21T22:50:15.965 INFO:teuthology.orchestra.run.smithi094.stdout:608 punch from 0x98a9669 to 0x98aa491, (0xe28 bytes) 2024-09-21T22:50:15.966 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:15.966 INFO:teuthology.orchestra.run.smithi094.stdout:610 punch from 0xa21a0d6 to 0xa2256c6, (0xb5f0 bytes) 2024-09-21T22:50:15.966 INFO:teuthology.orchestra.run.smithi094.stdout:611 write 0x8c12b1c thru 0x8c1a13b (0x7620 bytes) 2024-09-21T22:50:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:612 write 0x66f2fbf thru 0x66fdf49 (0xaf8b bytes) 2024-09-21T22:50:15.985 INFO:teuthology.orchestra.run.smithi094.stdout:613 read 0x87437e2 thru 0x8748c3e (0x545d bytes) 2024-09-21T22:50:16.000 INFO:teuthology.orchestra.run.smithi094.stdout:615 writesame 0xe0abd16 thru 0xe0b0a08 data_size 0x1(0x4cf3 bytes) 2024-09-21T22:50:16.000 INFO:teuthology.orchestra.run.smithi094.stdout:617 write 0x471c659 thru 0x4728c57 (0xc5ff bytes) 2024-09-21T22:50:16.010 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:16.005+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:16.011 INFO:teuthology.orchestra.run.smithi094.stdout:618 read 0x3230e73 thru 0x323623e (0x53cc bytes) 2024-09-21T22:50:16.018 INFO:teuthology.orchestra.run.smithi094.stdout:619 trunc from 0xe0b0a09 to 0x9ffbdb2 2024-09-21T22:50:16.039 INFO:teuthology.orchestra.run.smithi094.stdout:621 punch from 0x1a61672 to 0x1a6a757, (0x90e5 bytes) 2024-09-21T22:50:16.039 INFO:teuthology.orchestra.run.smithi094.stdout:622 writesame 0x95d3ac thru 0x964a5b data_size 0x48(0x76b0 bytes) 2024-09-21T22:50:16.047 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.047 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:16.065 INFO:teuthology.orchestra.run.smithi094.stdout:623 write 0xab72bfe thru 0xab7f799 (0xcb9c bytes) 2024-09-21T22:50:16.080 INFO:teuthology.orchestra.run.smithi094.stdout:624 clone 3 order 19 su 131072 sc 5 2024-09-21T22:50:16.111 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:16.106+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:16.182 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.182 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:16.212 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:16.206+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:16.313 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:16.310+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:16.358 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.358 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:16.413 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:16.410+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:16.490 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.490 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:16.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:16.510+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:16.615 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:16.609+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:16.625 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.625 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:16.715 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:16.710+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:16.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:16.803 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:16.803 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:16.815 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:16.809+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:16.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:16.916 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:16.917 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:16.917 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:16.914+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:16.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:16.934 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:16.950 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:16.967 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:16.967 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:16.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:16.988 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:711 3.0 e28/24 3:079e4dd3:::rbd_data.111ecbf88771.00000000000001b2:head v 28'7, mlcod=28'5)", 2024-09-21T22:50:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.2084325900000001, 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013689850000000001, 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.779996+0000", 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780009+0000", 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2391e-05 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780016+0000", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.126e-06 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780022+0000", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2299999999999996e-06 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780096+0000", 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3664999999999994e-05 2024-09-21T22:50:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.780122+0000", 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6571999999999999e-05 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.781333+0000", 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001211057 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.781365+0000", 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1439000000000002e-05 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:726 3.0 e28/24 3:07ca066a:::rbd_data.111ecbf88771.0000000000000134:head v 28'9, mlcod=28'6)", 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.1914056369999999, 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016183619999999999, 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797024+0000", 2024-09-21T22:50:16.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0599999999999999e-07 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797030+0000", 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4969999999999998e-06 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797044+0000", 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3868e-05 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797052+0000", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1869999999999997e-06 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797110+0000", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7995000000000003e-05 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797129+0000", 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9344e-05 2024-09-21T22:50:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798601+0000", 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014715419999999999 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798641+0000", 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0022999999999999e-05 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:966 3.0 e29/24 3:04479396:::rbd_id.image_client.0-clone2:head v 29'13, mlcod=28'11)", 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.104823+0000", 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.883605083, 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001432558, 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104823+0000", 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104823+0000", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104824+0000", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1800000000000001e-07 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104847+0000", 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3385999999999999e-05 2024-09-21T22:50:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104861+0000", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3465999999999999e-05 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104870+0000", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.149e-06 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.104966+0000", 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.624e-05 2024-09-21T22:50:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.105001+0000", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5126999999999998e-05 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.106147+0000", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001146036 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.106256+0000", 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108736 2024-09-21T22:50:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1020 3.0 e29/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 29'15, mlcod=28'12)", 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.822996753, 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018427630000000001, 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165434+0000", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165447+0000", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3157e-05 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165474+0000", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6621000000000001e-05 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165487+0000", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3529999999999999e-05 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165644+0000", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015642 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165712+0000", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8646000000000006e-05 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167238+0000", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001525804 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167274+0000", 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6329999999999999e-05 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1029 3.0 e29/24 3:04e5c2a8:::rbd_data.111ecbf88771.0000000000000070:head v 29'17, mlcod=29'13)", 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.169480+0000", 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.8189485940000001, 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014715329999999999, 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169480+0000", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169480+0000", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169481+0000", 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739999999999999e-06 2024-09-21T22:50:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169488+0000", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4420000000000001e-06 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169506+0000", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7265000000000002e-05 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169514+0000", 2024-09-21T22:50:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9840000000000001e-06 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169562+0000", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8594000000000003e-05 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169582+0000", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9635000000000001e-05 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.170829+0000", 2024-09-21T22:50:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001247427 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.170951+0000", 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000122012 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1030 3.0 e29/24 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head v 29'19, mlcod=29'13)", 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.8188937709999999, 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022031799999999999, 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5499999999999999e-07 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169544+0000", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9320000000000002e-06 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169551+0000", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3209999999999999e-06 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169557+0000", 2024-09-21T22:50:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5779999999999998e-06 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169973+0000", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00041665499999999999 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169994+0000", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0679999999999999e-05 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171635+0000", 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016405040000000001 2024-09-21T22:50:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171738+0000", 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103155 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1035 3.9 e29/24 3:94f6a430:::rbd_data.111ecbf88771.00000000000000d1:head v 29'5, mlcod=27'2)", 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.173435+0000", 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.814993308, 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001454728, 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173435+0000", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173435+0000", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173436+0000", 2024-09-21T22:50:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173453+0000", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6733999999999999e-05 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173475+0000", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2251000000000001e-05 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173490+0000", 2024-09-21T22:50:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5406000000000001e-05 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173563+0000", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2276999999999999e-05 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.173584+0000", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1342999999999999e-05 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.174853+0000", 2024-09-21T22:50:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012690749999999999 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.174890+0000", 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6602e-05 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1238 3.9 e29/24 3:912aea96:::rbd_data.111ea6ec99f1.00000000000001fa:head v 29'6, mlcod=28'3)", 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.712627+0000", 2024-09-21T22:50:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.2758010980000001, 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014869939999999999, 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712627+0000", 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712627+0000", 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712630+0000", 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712758+0000", 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012730599999999999 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712797+0000", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9569e-05 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712820+0000", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2436999999999999e-05 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712918+0000", 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8591999999999994e-05 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.712946+0000", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7906999999999999e-05 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714065+0000", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001118998 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714114+0000", 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9159000000000001e-05 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1239 3.9 e29/24 3:94979634:::rbd_data.111ea6ec99f1.00000000000001fb:head v 29'7, mlcod=28'3)", 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.714637+0000", 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.273791374, 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001358997, 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714637+0000", 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714637+0000", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714638+0000", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.057e-06 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714656+0000", 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.772e-05 2024-09-21T22:50:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714677+0000", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1253000000000001e-05 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714695+0000", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8352999999999999e-05 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714766+0000", 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0371999999999996e-05 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.714786+0000", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9945999999999998e-05 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.715953+0000", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001167127 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.715996+0000", 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3169e-05 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1246 3.9 e29/24 3:93836c00:::rbd_data.111ea6ec99f1.00000000000000bc:head v 29'8, mlcod=29'6)", 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.730529+0000", 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.257899466, 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013392650000000001, 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730529+0000", 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730529+0000", 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730531+0000", 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1600000000000001e-06 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730585+0000", 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3692e-05 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730613+0000", 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7860000000000001e-05 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730631+0000", 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8703999999999999e-05 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730710+0000", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9062999999999998e-05 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.730732+0000", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1590000000000002e-05 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.731820+0000", 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001087677 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.731868+0000", 2024-09-21T22:50:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8519000000000001e-05 2024-09-21T22:50:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1250 3.9 e29/24 3:912aea96:::rbd_data.111ea6ec99f1.00000000000001fa:head v 29'9, mlcod=29'7)", 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.741401+0000", 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.24702716, 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014634940000000001, 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741401+0000", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741401+0000", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741404+0000", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5500000000000001e-06 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741420+0000", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6526000000000001e-05 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741455+0000", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5153e-05 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741476+0000", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0417000000000001e-05 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741579+0000", 2024-09-21T22:50:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103331 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.741605+0000", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5884000000000002e-05 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.742813+0000", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001208097 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.742865+0000", 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1536000000000003e-05 2024-09-21T22:50:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1349 3.0 e29/24 3:05bd433a:::rbd_data.111ea6ec99f1.00000000000000bb:head v 29'21, mlcod=29'19)", 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1542645249999999, 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020573869999999999, 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834165+0000", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6190000000000001e-06 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834177+0000", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.189e-05 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834200+0000", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3110999999999999e-05 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834210+0000", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0009e-05 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834373+0000", 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016245800000000001 2024-09-21T22:50:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834430+0000", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7402999999999999e-05 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836171+0000", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017403659999999999 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836221+0000", 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0531000000000001e-05 2024-09-21T22:50:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1350 3.9 e29/24 3:93836c00:::rbd_data.111ea6ec99f1.00000000000000bc:head v 29'11, mlcod=29'9)", 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.834203+0000", 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1542250159999998, 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001350059, 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834203+0000", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834203+0000", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834204+0000", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5099999999999998e-07 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834215+0000", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0779e-05 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834227+0000", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1895e-05 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834234+0000", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8029999999999999e-06 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834334+0000", 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100116 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834354+0000", 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0066000000000001e-05 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.835441+0000", 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00108687 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.835553+0000", 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011257899999999999 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1363 3.4 e29/24 3:217488bc:::rbd_data.111ea6ec99f1.0000000000000153:head v 29'13, mlcod=29'11)", 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.842042+0000", 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1463861140000002, 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013590399999999999, 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842042+0000", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842042+0000", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842044+0000", 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-06 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842086+0000", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2104000000000002e-05 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842128+0000", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1956000000000003e-05 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842147+0000", 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9179999999999999e-05 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842246+0000", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8325000000000005e-05 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.842274+0000", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8534000000000002e-05 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.843295+0000", 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001020165 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.843401+0000", 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010675600000000001 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1848 3.4 e29/24 3:252aa742:::rbd_data.111ea6ec99f1.00000000000002cf:head v 29'14, mlcod=29'12)", 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8332049709999998, 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001883619, 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155226+0000", 2024-09-21T22:50:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155264+0000", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8418999999999999e-05 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155305+0000", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1029999999999998e-05 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155329+0000", 2024-09-21T22:50:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3623000000000001e-05 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155486+0000", 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015655899999999999 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155527+0000", 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1083e-05 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157019+0000", 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001492555 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157107+0000", 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7723e-05 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1853 3.4 e29/24 3:252aa742:::rbd_data.111ea6ec99f1.00000000000002cf:head v 29'15, mlcod=29'13)", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.163643+0000", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.824784932, 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014406849999999999, 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163643+0000", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163643+0000", 2024-09-21T22:50:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163646+0000", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5900000000000002e-06 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163666+0000", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.969e-05 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163707+0000", 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1801999999999997e-05 2024-09-21T22:50:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163727+0000", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9579999999999999e-05 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163795+0000", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8261000000000003e-05 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.163821+0000", 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6041999999999999e-05 2024-09-21T22:50:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.164959+0000", 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011374079999999999 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.165084+0000", 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012531200000000001 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1868 3.0 e29/24 3:031c5811:::rbd_data.111ea6ec99f1.00000000000001a4:head v 29'22, mlcod=29'20)", 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8041808449999999, 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016467299999999999, 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184250+0000", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6759999999999999e-06 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184311+0000", 2024-09-21T22:50:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0927000000000001e-05 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184344+0000", 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3376e-05 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184360+0000", 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5109999999999999e-05 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184459+0000", 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8964999999999999e-05 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184487+0000", 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8092000000000001e-05 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185841+0000", 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013539789999999999 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185894+0000", 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3604999999999999e-05 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2008 3.4 e29/24 3:27f59992:::rbd_data.111ea6ec99f1.000000000000037d:head v 29'16, mlcod=29'14)", 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.594720921, 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016704179999999999, 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393709+0000", 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0729999999999999e-06 2024-09-21T22:50:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393727+0000", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7227e-05 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393753+0000", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6066000000000001e-05 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393769+0000", 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.609e-05 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393939+0000", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017003600000000001 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.394002+0000", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3504999999999996e-05 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395320+0000", 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001317599 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395378+0000", 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7822000000000003e-05 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2068 3.4 e29/24 3:27f59992:::rbd_data.111ea6ec99f1.000000000000037d:head v 29'17, mlcod=29'15)", 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.489793+0000", 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.4986357699999999, 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001573248, 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489793+0000", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489793+0000", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489795+0000", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5469999999999999e-06 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489812+0000", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6416e-05 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489854+0000", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2796999999999997e-05 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.489874+0000", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.001e-05 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.490008+0000", 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133802 2024-09-21T22:50:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.490046+0000", 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7765999999999997e-05 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.491222+0000", 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011761009999999999 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.491366+0000", 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000143809 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2114 3.9 e29/24 3:91282459:::rbd_data.111ea6ec99f1.00000000000002fb:head v 29'13, mlcod=29'11)", 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.4255566750000002, 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001640273, 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562874+0000", 2024-09-21T22:50:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.03e-06 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562894+0000", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0752999999999999e-05 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562914+0000", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9987e-05 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562926+0000", 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1153e-05 2024-09-21T22:50:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563053+0000", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127687 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563093+0000", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9646e-05 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564463+0000", 2024-09-21T22:50:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013701080000000001 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564512+0000", 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8909000000000002e-05 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.075 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:17.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:17.077 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:17.077 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:17.077 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:17.078 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:17.078 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:17.079 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:17.079 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:17.079 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:17.074+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:17.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:50:17.177 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:17.173+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:17.178 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone2 from 0xab7f79a (overlap 0x232e26) to 0x0 2024-09-21T22:50:17.252 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:17.253 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:17.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:17.317 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:17.313+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:17.318 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T22:50:17.338 INFO:teuthology.orchestra.run.smithi094.stdout:626 writesame 0xb243448 thru 0xb2505c7 data_size 0x40(0xd180 bytes) 2024-09-21T22:50:17.338 INFO:teuthology.orchestra.run.smithi094.stdout:627 writesame 0x80829ff thru 0x80861e1 data_size 0x13(0x37e3 bytes) 2024-09-21T22:50:17.342 INFO:teuthology.orchestra.run.smithi094.stdout:628 write 0x320c369 thru 0x3216b20 (0xa7b8 bytes) 2024-09-21T22:50:17.348 INFO:teuthology.orchestra.run.smithi094.stdout:631 read 0x17941de thru 0x179ecfa (0xab1d bytes) 2024-09-21T22:50:17.348 INFO:teuthology.orchestra.run.smithi094.stdout:632 read 0x17cb915 thru 0x17d3e20 (0x850c bytes) 2024-09-21T22:50:17.348 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:17.358 INFO:teuthology.orchestra.run.smithi094.stdout:637 trunc from 0xb2505c8 to 0x395a1f4 2024-09-21T22:50:17.374 INFO:teuthology.orchestra.run.smithi094.stdout:638 read 0x28b9567 thru 0x28c845a (0xeef4 bytes) 2024-09-21T22:50:17.374 INFO:teuthology.orchestra.run.smithi094.stdout:639 read 0x281a604 thru 0x28217c1 (0x71be bytes) 2024-09-21T22:50:17.375 INFO:teuthology.orchestra.run.smithi099.stdout: marking collection for removal 2024-09-21T22:50:17.375 INFO:teuthology.orchestra.run.smithi099.stdout:setting '_remove' omap key 2024-09-21T22:50:17.375 INFO:teuthology.orchestra.run.smithi099.stdout:finish_remove_pgs 2.7_head removing 2.7 2024-09-21T22:50:17.376 INFO:teuthology.orchestra.run.smithi099.stdout:Remove successful 2024-09-21T22:50:17.379 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:17.374+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:17.396 INFO:teuthology.orchestra.run.smithi094.stdout:640 write 0xd1e2d73 thru 0xd1e97af (0x6a3d bytes) 2024-09-21T22:50:17.401 INFO:teuthology.orchestra.run.smithi094.stdout:642 writesame 0x4b0e871 thru 0x4b11915 data_size 0x7(0x30a5 bytes) 2024-09-21T22:50:17.405 INFO:teuthology.orchestra.run.smithi094.stdout:643 read 0x80edf3f thru 0x80fb830 (0xd8f2 bytes) 2024-09-21T22:50:17.405 INFO:teuthology.orchestra.run.smithi094.stdout:644 writesame 0x1b0bd0d thru 0x1b0f339 data_size 0x43(0x362d bytes) 2024-09-21T22:50:17.410 INFO:teuthology.orchestra.run.smithi094.stdout:645 write 0x2b894de thru 0x2b92b8e (0x96b1 bytes) 2024-09-21T22:50:17.415 INFO:teuthology.orchestra.run.smithi094.stdout:646 read 0x7fe2fbd thru 0x7fe836c (0x53b0 bytes) 2024-09-21T22:50:17.415 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:17.415 INFO:teuthology.orchestra.run.smithi094.stdout:649 read 0x24405e thru 0x24cf09 (0x8eac bytes) 2024-09-21T22:50:17.416 INFO:teuthology.orchestra.run.smithi094.stdout:650 read 0x12b701e thru 0x12c6522 (0xf505 bytes) 2024-09-21T22:50:17.416 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:17.416 INFO:teuthology.orchestra.run.smithi094.stdout:653 write 0x55d54b6 thru 0x55dfc5d (0xa7a8 bytes) 2024-09-21T22:50:17.419 INFO:teuthology.orchestra.run.smithi094.stdout:654 write 0x4b1d8e7 thru 0x4b2798f (0xa0a9 bytes) 2024-09-21T22:50:17.424 INFO:teuthology.orchestra.run.smithi094.stdout:658 punch from 0x7e7b707 to 0x7e83ffc, (0x88f5 bytes) 2024-09-21T22:50:17.424 INFO:teuthology.orchestra.run.smithi094.stdout:659 read 0x6c436f6 thru 0x6c49849 (0x6154 bytes) 2024-09-21T22:50:17.424 INFO:teuthology.orchestra.run.smithi094.stdout:661 write 0x5752187 thru 0x575e8fb (0xc775 bytes) 2024-09-21T22:50:17.429 INFO:teuthology.orchestra.run.smithi094.stdout:662 clone 4 order 19 su 65536 sc 8 2024-09-21T22:50:17.459 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:17.459 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:17.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:17.480 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:17.478+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:17.581 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:17.577+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:17.641 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:615 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9637240919999996, 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011471453, 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 615 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673350+0000", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5199999999999996e-07 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673353+0000", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.864e-06 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673362+0000", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6640000000000004e-06 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673367+0000", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7400000000000004e-06 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.676392+0000", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030256549999999999 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.676572+0000", 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017976900000000001 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.677171+0000", 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059889800000000005 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680874+0000", 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037028819999999998 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684627+0000", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037529310000000001 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684720+0000", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3356000000000005e-05 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684821+0000", 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101042 2024-09-21T22:50:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:616 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9636855779999998, 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012626187000000001, 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 616 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673388+0000", 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673389+0000", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5400000000000001e-07 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673391+0000", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7180000000000001e-06 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673400+0000", 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6959999999999994e-06 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673405+0000", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8439999999999997e-06 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.677910+0000", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045045809999999997 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.678042+0000", 2024-09-21T22:50:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013247500000000001 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.678602+0000", 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00055957299999999995 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684502+0000", 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059001110000000004 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685933+0000", 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001431444 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685970+0000", 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6600999999999997e-05 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686014+0000", 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4589999999999998e-05 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:617 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9636613140000003, 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012746733999999999, 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 617 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673412+0000", 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673413+0000", 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000005e-07 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673416+0000", 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5900000000000002e-06 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673423+0000", 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7360000000000001e-06 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673428+0000", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5190000000000003e-06 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.679721+0000", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0062929830000000003 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.679840+0000", 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011886 2024-09-21T22:50:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680378+0000", 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053876300000000005 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686031+0000", 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056522059999999999 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686076+0000", 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5603999999999999e-05 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686115+0000", 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9156000000000003e-05 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686159+0000", 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3832999999999999e-05 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:618 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9636396989999998, 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012878603000000001, 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 618 2024-09-21T22:50:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673434+0000", 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673435+0000", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.46e-07 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673437+0000", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.52e-06 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673445+0000", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3599999999999996e-06 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673450+0000", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.634e-06 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.680924+0000", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074742860000000001 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.681001+0000", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7169999999999995e-05 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.681342+0000", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000340281 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686038+0000", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046963689999999997 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686207+0000", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168868 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686269+0000", 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2445000000000003e-05 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686313+0000", 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3223999999999998e-05 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:619 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9636174549999996, 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013060782, 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 619 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673457+0000", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.58e-07 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673459+0000", 2024-09-21T22:50:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6539999999999999e-06 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673465+0000", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.02e-06 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673470+0000", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4479999999999996e-06 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682015+0000", 2024-09-21T22:50:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085449849999999997 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682193+0000", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017775199999999999 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.683339+0000", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011463549999999999 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686337+0000", 2024-09-21T22:50:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00299797 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686383+0000", 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6462999999999999e-05 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686467+0000", 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3796999999999997e-05 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686517+0000", 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9879999999999997e-05 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:620 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9635980069999999, 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014257994, 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 620 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673479+0000", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5560000000000002e-06 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673495+0000", 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.615e-05 2024-09-21T22:50:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673500+0000", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9439999999999996e-06 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684881+0000", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011381242 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684990+0000", 2024-09-21T22:50:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109197 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685289+0000", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029849100000000001 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687532+0000", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022436510000000002 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687588+0000", 2024-09-21T22:50:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5655999999999997e-05 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687652+0000", 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3544e-05 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687734+0000", 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2126999999999995e-05 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:621 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9635673430000002, 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018613351, 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 621 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673507+0000", 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673510+0000", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8059999999999999e-06 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673516+0000", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3099999999999997e-06 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673520+0000", 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4619999999999998e-06 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686563+0000", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013042876 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686663+0000", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9570999999999999e-05 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687059+0000", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039618800000000001 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687883+0000", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082348899999999995 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692061+0000", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004178022 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692094+0000", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3473999999999998e-05 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692120+0000", 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5711000000000002e-05 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:680 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.89089098, 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013778161000000001, 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 680 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746187+0000", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9160000000000003e-06 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746193+0000", 2024-09-21T22:50:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746200+0000", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5760000000000002e-06 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753332+0000", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071319840000000001 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753409+0000", 2024-09-21T22:50:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7111000000000001e-05 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753698+0000", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028885599999999999 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754451+0000", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075351400000000005 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759904+0000", 2024-09-21T22:50:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054529679999999999 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759937+0000", 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3018999999999997e-05 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759961+0000", 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3889000000000001e-05 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1045 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4538408980000002, 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011667999, 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1045 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183233+0000", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183234+0000", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2300000000000001e-06 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183241+0000", 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4699999999999999e-06 2024-09-21T22:50:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183259+0000", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8133000000000001e-05 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183285+0000", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6083000000000001e-05 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.189608+0000", 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063234750000000003 2024-09-21T22:50:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.189839+0000", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230881 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.190878+0000", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001038509 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194705+0000", 2024-09-21T22:50:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038270959999999999 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194730+0000", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5534e-05 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194809+0000", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8337000000000006e-05 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194901+0000", 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2250999999999995e-05 2024-09-21T22:50:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1046 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4537713290000003, 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015794288, 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1046 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183304+0000", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1930000000000001e-06 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183311+0000", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3339999999999996e-06 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183339+0000", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8030000000000001e-05 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183346+0000", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.192825+0000", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094795499999999998 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193017+0000", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000192287 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193797+0000", 2024-09-21T22:50:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000779388 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197391+0000", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035944969999999999 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198914+0000", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015223039999999999 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199027+0000", 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113127 2024-09-21T22:50:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199097+0000", 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0024999999999994e-05 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1047 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4537187039999999, 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017348148000000001, 2024-09-21T22:50:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1047 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183356+0000", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183362+0000", 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.806e-06 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183371+0000", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5529999999999998e-06 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183378+0000", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8689999999999998e-06 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194941+0000", 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011562767999999999 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195056+0000", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115281 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195493+0000", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043665799999999998 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199120+0000", 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036272919999999998 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200347+0000", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001226782 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200566+0000", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021947500000000001 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200703+0000", 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013704200000000001 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1048 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4536863789999996, 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017543792999999998, 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1048 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183388+0000", 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-21T22:50:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183392+0000", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7139999999999999e-06 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183401+0000", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2469999999999997e-06 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183408+0000", 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0469999999999996e-06 2024-09-21T22:50:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195923+0000", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012514658 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196021+0000", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8113000000000001e-05 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196394+0000", 2024-09-21T22:50:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037272199999999997 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200738+0000", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043443179999999998 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200803+0000", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5166000000000006e-05 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200869+0000", 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6190000000000002e-05 2024-09-21T22:50:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200931+0000", 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1996999999999996e-05 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1049 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4536567439999999, 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017955736999999999, 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1049 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183418+0000", 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183421+0000", 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5580000000000001e-06 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183431+0000", 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3430000000000002e-06 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183438+0000", 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197463+0000", 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014025892999999999 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197596+0000", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013223600000000001 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198242+0000", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064663999999999998 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201081+0000", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002839144 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201120+0000", 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8563000000000003e-05 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201262+0000", 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014153000000000001 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201373+0000", 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111236 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1050 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4536268840000002, 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020333185, 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1050 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183448+0000", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183451+0000", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6389999999999998e-06 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183460+0000", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183467+0000", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9600000000000003e-06 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199154+0000", 2024-09-21T22:50:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015686734000000001 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199259+0000", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105364 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199748+0000", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048820500000000002 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203483+0000", 2024-09-21T22:50:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003735787 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203519+0000", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5954000000000002e-05 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203653+0000", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013328799999999999 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203780+0000", 2024-09-21T22:50:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012739399999999999 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1051 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4535735020000002, 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021843896000000002, 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1051 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183501+0000", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2399999999999997e-07 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183506+0000", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183527+0000", 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1353e-05 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183535+0000", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0180000000000001e-06 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201414+0000", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01787888 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201496+0000", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2201999999999997e-05 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201824+0000", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032739100000000001 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205098+0000", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032737280000000001 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205130+0000", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2805000000000002e-05 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205258+0000", 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127911 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205344+0000", 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5882000000000002e-05 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1052 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4535256759999999, 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022039640999999999, 2024-09-21T22:50:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1052 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183549+0000", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183553+0000", 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1130000000000003e-06 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183563+0000", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6930000000000002e-06 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183570+0000", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202302+0000", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018731863000000001 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202413+0000", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111117 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202922+0000", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050897300000000002 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205382+0000", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024604240000000001 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205403+0000", 2024-09-21T22:50:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1058999999999999e-05 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205503+0000", 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100068 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205588+0000", 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4584000000000002e-05 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1053 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4534945859999997, 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022880587000000001, 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1053 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183580+0000", 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-21T22:50:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183592+0000", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.224e-05 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183602+0000", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094e-05 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183609+0000", 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1810000000000004e-06 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203845+0000", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020235907000000001 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203896+0000", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1187999999999998e-05 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204112+0000", 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021558999999999999 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205609+0000", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001496743 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206370+0000", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076130800000000004 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206421+0000", 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0491000000000002e-05 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206460+0000", 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9218999999999998e-05 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1054 3.2 3:46641ad1:::rbd_data.111ecbf88771.0000000000000000:head [stat,truncate 204800] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4492977749999998, 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019736885999999999, 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1054 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187777+0000", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187783+0000", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2780000000000001e-06 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187794+0000", 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1464999999999999e-05 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187811+0000", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6716999999999999e-05 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204451+0000", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016639856000000001 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204504+0000", 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3763e-05 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204604+0000", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9433999999999997e-05 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205621+0000", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010175049999999999 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207340+0000", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718884 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207409+0000", 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8959000000000002e-05 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207513+0000", 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010366600000000001 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1805 3.b 3:d5e4ffc8:::rbd_data.111ea6ec99f1.000000000000009d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.578332589, 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.032681931999999997, 2024-09-21T22:50:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1805 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058743+0000", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.37e-06 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058751+0000", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058769+0000", 2024-09-21T22:50:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7927e-05 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058780+0000", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1353e-05 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058934+0000", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015403899999999999 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.059298+0000", 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036453700000000001 2024-09-21T22:50:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065146+0000", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058479789999999997 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091319+0000", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026172104000000002 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091334+0000", 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4939999999999999e-05 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091379+0000", 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5593000000000002e-05 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091423+0000", 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4014000000000002e-05 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3139 3.7 3:e732905d:::rbd_data.111ea6ec99f1.0000000000000015:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:17.192380+0000", 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.44469362699999998, 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012079236, 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3139 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192380+0000", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192380+0000", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192382+0000", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192390+0000", 2024-09-21T22:50:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0620000000000001e-06 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192419+0000", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8289999999999998e-05 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192430+0000", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0957e-05 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192503+0000", 2024-09-21T22:50:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3224000000000002e-05 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192586+0000", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3452e-05 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.192770+0000", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183547 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.193931+0000", 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011611150000000001 2024-09-21T22:50:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.204335+0000", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01040407 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.204409+0000", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3879000000000002e-05 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.204459+0000", 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0565000000000001e-05 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:17.752 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:17.753 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:17.753 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:17.750+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:17.854 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:17.850+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:17.885 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:17.886 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:17.886 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:17.886 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:17.899 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:17.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.7.6 2024-09-21T22:50:17.955 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:17.953+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:18.055 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:18.049+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:18.083 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:18.083 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:18.084 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:18.084 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:18.084 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:18.087 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:50:18.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:18.156 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:18.150+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:18.190 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone3 from 0xd1e97b0 (overlap 0x395a1f4) to 0x0 2024-09-21T22:50:18.240 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:451 3.1a 3:58b09410:::rbd_data.111ecbf88771.0000000000000072:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.7644245640000005, 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074162789999999996, 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 451 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469994+0000", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.626e-06 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470004+0000", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4159999999999993e-06 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470020+0000", 2024-09-21T22:50:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6447e-05 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470031+0000", 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470093+0000", 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2024000000000001e-05 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470206+0000", 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113238 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.471324+0000", 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001117364 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477262+0000", 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059383550000000002 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477344+0000", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1831999999999998e-05 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477378+0000", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3618000000000002e-05 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477409+0000", 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1263000000000002e-05 2024-09-21T22:50:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:452 3.a 3:56d9bfad:::rbd_data.111ecbf88771.0000000000000073:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.7643105400000003, 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072291539999999998, 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 452 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470108+0000", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.08e-06 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470114+0000", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470124+0000", 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0465999999999999e-05 2024-09-21T22:50:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470132+0000", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0670000000000003e-06 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470202+0000", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0653000000000003e-05 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470412+0000", 2024-09-21T22:50:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209817 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477222+0000", 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068098330000000004 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477263+0000", 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0694e-05 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477336+0000", 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3250999999999994e-05 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:653 3.a 3:551485c8:::rbd_data.111ecbf88771.00000000000000fa:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.512737692, 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082106869999999995, 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 653 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721681+0000", 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5689999999999999e-06 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721689+0000", 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721709+0000", 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9343000000000001e-05 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721718+0000", 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4150000000000005e-06 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721783+0000", 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535000000000005e-05 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721876+0000", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3410000000000002e-05 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729625+0000", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077490689999999999 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729751+0000", 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126062 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729890+0000", 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013910300000000001 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1008 3.14 3:2c7bff05:::rbd_data.111ecbf88771.0000000000000030:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0754865530000002, 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068041639999999997, 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1008 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158933+0000", 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158945+0000", 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158980+0000", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4980000000000001e-05 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159000+0000", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0015999999999999e-05 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159308+0000", 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030743099999999999 2024-09-21T22:50:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159539+0000", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023104500000000001 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.160019+0000", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048036299999999998 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.162010+0000", 2024-09-21T22:50:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019912760000000002 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165428+0000", 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034175820000000002 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165584+0000", 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015635900000000001 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165735+0000", 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015054900000000001 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1033 3.1b e29/24 3:dd3d94e9:::rbd_data.111ecbf88771.0000000000000074:head v 29'9, mlcod=28'6)", 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0610121939999999, 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070721250000000003, 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173407+0000", 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6759999999999999e-06 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173420+0000", 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3224000000000001e-05 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173446+0000", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6118999999999998e-05 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173465+0000", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8542999999999998e-05 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173677+0000", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000212936 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173731+0000", 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3981999999999999e-05 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180381+0000", 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066499920000000004 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180477+0000", 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5653000000000001e-05 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:2014 3.1d e29/24 3:bb619233:::rbd_object_map.111ea6ec99f1:head v 29'189, mlcod=29'184)", 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.834174602, 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076346629999999999, 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4899999999999995e-07 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400249+0000", 2024-09-21T22:50:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6119999999999998e-06 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400262+0000", 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2965e-05 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400269+0000", 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0360000000000001e-06 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400349+0000", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0105999999999995e-05 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400368+0000", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8749e-05 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407812+0000", 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074442179999999998 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407877+0000", 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5427999999999998e-05 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:2015 3.1d e29/24 3:bb619233:::rbd_object_map.111ea6ec99f1:head v 29'190, mlcod=29'186)", 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.401173+0000", 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8332438880000002, 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067477209999999999, 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401173+0000", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401173+0000", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401174+0000", 2024-09-21T22:50:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2800000000000005e-07 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401181+0000", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4359999999999999e-06 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401194+0000", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3151e-05 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401201+0000", 2024-09-21T22:50:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4070000000000004e-06 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401242+0000", 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0312000000000003e-05 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401264+0000", 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2534000000000001e-05 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407887+0000", 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006622953 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407921+0000", 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4e-05 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3027 3.3 3:c2ee4b09:::rbd_header.111ea6ec99f1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.153235915, 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071998269999999998, 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3027 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081185+0000", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1729999999999997e-06 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081204+0000", 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8940000000000002e-05 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081244+0000", 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0092000000000002e-05 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081264+0000", 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9344e-05 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081411+0000", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014733500000000001 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081421+0000", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6250000000000002e-06 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.085977+0000", 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045564969999999996 2024-09-21T22:50:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.086051+0000", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3627999999999994e-05 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.086406+0000", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035498300000000001 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.087041+0000", 2024-09-21T22:50:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063464299999999995 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088313+0000", 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001272872 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088350+0000", 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6795999999999998e-05 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088381+0000", 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0898999999999997e-05 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3128 3.13 e31/24 3:cd5340d6:::rbd_data.111ea6ec99f1.0000000000000005:head v 31'39, mlcod=30'36)", 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0495488589999999, 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077291139999999996, 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184870+0000", 2024-09-21T22:50:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7680000000000001e-06 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184884+0000", 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4115e-05 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184911+0000", 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6797000000000001e-05 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184924+0000", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2828e-05 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185057+0000", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133287 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185105+0000", 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7573e-05 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192499+0000", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073943059999999998 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192597+0000", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999999e-05 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3125 3.11 e31/24 3:8b63553f:::rbd_data.111ea6ec99f1.0000000000000002:head v 31'40, mlcod=30'37)", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.049376138, 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075503890000000002, 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185042+0000", 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185050+0000", 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8279999999999992e-06 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185071+0000", 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0726000000000001e-05 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185089+0000", 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8280000000000001e-05 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185200+0000", 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110717 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185251+0000", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0930000000000002e-05 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192476+0000", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072249539999999996 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192591+0000", 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115382 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3127 3.1 e31/24 3:86733ef3:::rbd_data.111ea6ec99f1.0000000000000004:head v 31'24, mlcod=30'21)", 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0488637139999999, 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071006139999999999, 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185555+0000", 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.592e-06 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185568+0000", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2787e-05 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185589+0000", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1373e-05 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185605+0000", 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5381999999999999e-05 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185742+0000", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013768400000000001 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185806+0000", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3942999999999994e-05 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192598+0000", 2024-09-21T22:50:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067914309999999997 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192654+0000", 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6422000000000003e-05 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3136 3.9 3:901d0113:::rbd_data.111ea6ec99f1.000000000000000d:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.193744+0000", 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.040673228, 2024-09-21T22:50:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067041180000000002, 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3136 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193744+0000", 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193744+0000", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193747+0000", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5670000000000002e-06 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193758+0000", 2024-09-21T22:50:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1223000000000001e-05 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193793+0000", 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5193e-05 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193815+0000", 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2223000000000001e-05 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193964+0000", 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014876900000000001 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.194195+0000", 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023129800000000001 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.194708+0000", 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00051312899999999997 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199759+0000", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050510169999999997 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200303+0000", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054359700000000005 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200380+0000", 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6687000000000006e-05 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200448+0000", 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8415000000000001e-05 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3138 3.3 3:c00cbbd7:::rbd_data.111ea6ec99f1.000000000000000f:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.193860+0000", 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.040557059, 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066288889999999998, 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3138 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193860+0000", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193860+0000", 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193862+0000", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193870+0000", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1070000000000005e-06 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193893+0000", 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2727000000000001e-05 2024-09-21T22:50:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193909+0000", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6575999999999999e-05 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.194042+0000", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013281999999999999 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.194251+0000", 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020935600000000001 2024-09-21T22:50:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.194703+0000", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045207599999999999 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.197672+0000", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029687419999999999 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200226+0000", 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025543940000000002 2024-09-21T22:50:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200359+0000", 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013212900000000001 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.200489+0000", 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013039200000000001 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3140 3.9 3:94fc2d77:::rbd_data.111ea6ec99f1.0000000000000018:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.040470676, 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090689740000000005, 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3140 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193948+0000", 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4929999999999999e-06 2024-09-21T22:50:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193954+0000", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193975+0000", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1256000000000001e-05 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193990+0000", 2024-09-21T22:50:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4714e-05 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.197704+0000", 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003714326 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.197947+0000", 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242298 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.198405+0000", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045871900000000001 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199885+0000", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001479478 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.202762+0000", 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028774680000000002 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.202890+0000", 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012720599999999999 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203015+0000", 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012587899999999999 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3147 3.1b e31/24 3:dea0d918:::rbd_data.111ea6ec99f1.0000000000000028:head v 31'29, mlcod=30'26)", 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0347095369999999, 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070594120000000002, 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199710+0000", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.967e-06 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199724+0000", 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.394e-05 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199755+0000", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1962999999999999e-05 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199773+0000", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7955999999999999e-05 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199936+0000", 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000162602 2024-09-21T22:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199996+0000", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0130000000000002e-05 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.206664+0000", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066681650000000002 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.206767+0000", 2024-09-21T22:50:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010268900000000001 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3151 3.13 e31/24 3:cce47d61:::rbd_data.111ea6ec99f1.0000000000000069:head v 31'41, mlcod=30'37)", 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.030911173, 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072897400000000003, 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203508+0000", 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9039999999999999e-06 2024-09-21T22:50:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203524+0000", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5806000000000001e-05 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203559+0000", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5206000000000001e-05 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203581+0000", 2024-09-21T22:50:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2350000000000001e-05 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203701+0000", 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119287 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203756+0000", 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5745000000000002e-05 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.210549+0000", 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067928470000000003 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.210796+0000", 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024659500000000001 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3155 3.1b e31/24 3:dcf1d737:::rbd_data.111ea6ec99f1.000000000000011b:head v 31'31, mlcod=30'26)", 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.030202225, 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087106030000000008, 2024-09-21T22:50:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204217+0000", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204231+0000", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4613000000000001e-05 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204259+0000", 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7682999999999999e-05 2024-09-21T22:50:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204279+0000", 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9599999999999999e-05 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204760+0000", 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048154300000000002 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204816+0000", 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.63e-05 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.212749+0000", 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079325850000000007 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.212926+0000", 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017649899999999999 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3171 3.1a 3:5f8bd33a:::rbd_data.111ea6ec99f1.0000000000000210:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0210533589999999, 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087212890000000001, 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3171 2024-09-21T22:50:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213366+0000", 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.827e-06 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213374+0000", 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213404+0000", 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9799999999999999e-05 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213418+0000", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3944999999999999e-05 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.214844+0000", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014259940000000001 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.215033+0000", 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189571 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.215421+0000", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038801200000000002 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217787+0000", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002365818 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221880+0000", 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040925290000000001 2024-09-21T22:50:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221996+0000", 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115919 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.222085+0000", 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9351999999999994e-05 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3178 3.3 3:c64ebcdf:::rbd_data.111ea6ec99f1.0000000000000248:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.217604+0000", 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.016812931, 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067235990000000002, 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3178 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217604+0000", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217604+0000", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217607+0000", 2024-09-21T22:50:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8090000000000001e-06 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217620+0000", 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2493999999999999e-05 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217681+0000", 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1144999999999998e-05 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217702+0000", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.126e-05 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217829+0000", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012667399999999999 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.218064+0000", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023533000000000001 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.218449+0000", 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038480699999999998 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221712+0000", 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032634119999999998 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.224119+0000", 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024064910000000002 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.224227+0000", 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001083 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.224328+0000", 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100877 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3183 3.14 3:29e0d94b:::rbd_data.111ea6ec99f1.00000000000002b0:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0141351169999999, 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071140329999999996, 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3183 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220285+0000", 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7259999999999998e-06 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221056+0000", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077125900000000005 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221094+0000", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7991000000000003e-05 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221116+0000", 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2422999999999999e-05 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221251+0000", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134592 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221438+0000", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000187023 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221828+0000", 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038974099999999998 2024-09-21T22:50:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.226593+0000", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047647169999999999 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227227+0000", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063445899999999998 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227316+0000", 2024-09-21T22:50:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8633000000000003e-05 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227396+0000", 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0469000000000005e-05 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:18.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:18.347 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:18.341+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:18.395 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T22:50:18.398 INFO:teuthology.orchestra.run.smithi094.stdout:663 punch from 0x94351cc to 0x943ddde, (0x8c12 bytes) 2024-09-21T22:50:18.403 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.403 INFO:teuthology.orchestra.run.smithi094.stdout:667 write 0xd1c6d22 thru 0xd1d2c12 (0xbef1 bytes) 2024-09-21T22:50:18.420 INFO:teuthology.orchestra.run.smithi094.stdout:668 trunc from 0xd1e97b0 to 0x74e046e 2024-09-21T22:50:18.447 INFO:teuthology.orchestra.run.smithi094.stdout:669 read 0x4fee142 thru 0x4ff45cf (0x648e bytes) 2024-09-21T22:50:18.447 INFO:teuthology.orchestra.run.smithi094.stdout:670 write 0x3dd1bfe thru 0x3de1741 (0xfb44 bytes) 2024-09-21T22:50:18.447 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:18.441+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:18.454 INFO:teuthology.orchestra.run.smithi094.stdout:671 read 0x5d5b191 thru 0x5d68e43 (0xdcb3 bytes) 2024-09-21T22:50:18.454 INFO:teuthology.orchestra.run.smithi094.stdout:674 read 0xaf0dc3 thru 0xaf92b5 (0x84f3 bytes) 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:675 read 0x4048436 thru 0x40566e1 (0xe2ac bytes) 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:680 punch from 0x740ad23 to 0x740ce35, (0x2112 bytes) 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:685 punch from 0x1e2d0ff to 0x1e3c17e, (0xf07f bytes) 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:686 punch from 0x71faa64 to 0x7200743, (0x5cdf bytes) 2024-09-21T22:50:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:687 read 0x4fd6085 thru 0x4fdfe88 (0x9e04 bytes) 2024-09-21T22:50:18.461 INFO:teuthology.orchestra.run.smithi094.stdout:688 trunc from 0x74e046e to 0xbd0050e 2024-09-21T22:50:18.469 INFO:teuthology.orchestra.run.smithi094.stdout:689 read 0x2c1cf8a thru 0x2c1d7a1 (0x818 bytes) 2024-09-21T22:50:18.470 INFO:teuthology.orchestra.run.smithi094.stdout:690 write 0x73f689a thru 0x7404931 (0xe098 bytes) 2024-09-21T22:50:18.474 INFO:teuthology.orchestra.run.smithi094.stdout:691 write 0x9901293 thru 0x990c58d (0xb2fb bytes) 2024-09-21T22:50:18.478 INFO:teuthology.orchestra.run.smithi094.stdout:692 write 0xa51b354 thru 0xa528d6d (0xda1a bytes) 2024-09-21T22:50:18.484 INFO:teuthology.orchestra.run.smithi094.stdout:695 writesame 0x139dbf3 thru 0x13a66c8 data_size 0xd(0x8ad6 bytes) 2024-09-21T22:50:18.493 INFO:teuthology.orchestra.run.smithi094.stdout:696 read 0xab93858 thru 0xaba34f7 (0xfca0 bytes) 2024-09-21T22:50:18.501 INFO:teuthology.orchestra.run.smithi094.stdout:697 trunc from 0xbd0050e to 0x5d72193 2024-09-21T22:50:18.534 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:18.534 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:18.534 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:18.534 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:18.536 INFO:teuthology.orchestra.run.smithi094.stdout:698 write 0xa69434 thru 0xa6caa8 (0x3675 bytes) 2024-09-21T22:50:18.546 INFO:teuthology.orchestra.run.smithi094.stdout:699 write 0x22b74cc thru 0x22c13f8 (0x9f2d bytes) 2024-09-21T22:50:18.548 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:18.541+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:18.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:18.564 INFO:teuthology.orchestra.run.smithi094.stdout:700 write 0x7ba56ac thru 0x7ba9183 (0x3ad8 bytes) 2024-09-21T22:50:18.577 INFO:teuthology.orchestra.run.smithi094.stdout:701 trunc from 0x7ba9184 to 0xff7a5f 2024-09-21T22:50:18.613 INFO:teuthology.orchestra.run.smithi094.stdout:702 trunc from 0xff7a5f to 0x8656b6f 2024-09-21T22:50:18.621 INFO:teuthology.orchestra.run.smithi094.stdout:703 punch from 0x4164454 to 0x4165fbb, (0x1b67 bytes) 2024-09-21T22:50:18.621 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.622 INFO:teuthology.orchestra.run.smithi094.stdout:709 write 0x369186a thru 0x3697bcf (0x6366 bytes) 2024-09-21T22:50:18.635 INFO:teuthology.orchestra.run.smithi094.stdout:710 trunc from 0x8656b6f to 0x68c547 2024-09-21T22:50:18.649 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:18.645+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:18.681 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:18.682 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:18.682 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:18.682 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:18.682 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:18.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:18.706 INFO:teuthology.orchestra.run.smithi094.stdout:711 write 0xdad86f5 thru 0xdad9f06 (0x1812 bytes) 2024-09-21T22:50:18.712 INFO:teuthology.orchestra.run.smithi094.stdout:712 punch from 0x5452fa6 to 0x545f27d, (0xc2d7 bytes) 2024-09-21T22:50:18.739 INFO:teuthology.orchestra.run.smithi094.stdout:714 trunc from 0xdad9f07 to 0x57498c 2024-09-21T22:50:18.750 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:18.746+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:18.797 INFO:teuthology.orchestra.run.smithi094.stdout:715 write 0xa686ccd thru 0xa687bb0 (0xee4 bytes) 2024-09-21T22:50:18.812 INFO:teuthology.orchestra.run.smithi094.stdout:716 trunc from 0xa687bb1 to 0x611ffa8 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.843951721, 2024-09-21T22:50:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012130994000000001, 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4269", 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/4252523962", 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971598+0000", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1870000000000003e-06 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971609+0000", 2024-09-21T22:50:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0783e-05 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971644+0000", 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.472e-05 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971670+0000", 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6007e-05 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971856+0000", 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018626800000000001 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971893+0000", 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807000000000002e-05 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.975231+0000", 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00333811 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.975253+0000", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2160000000000001e-05 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979419+0000", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041659879999999998 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979758+0000", 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033847700000000001 2024-09-21T22:50:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979998+0000", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000240688 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.981947+0000", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001949002 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983505+0000", 2024-09-21T22:50:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015578409999999999 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983632+0000", 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126823 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983720+0000", 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8133000000000005e-05 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1188 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1844542919999999, 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085004539999999993, 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1188 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631088+0000", 2024-09-21T22:50:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1030000000000001e-06 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631094+0000", 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5570000000000001e-06 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631107+0000", 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2816000000000001e-05 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631115+0000", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.632917+0000", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001802056 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.633061+0000", 2024-09-21T22:50:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000143759 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.633680+0000", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061894499999999998 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639319+0000", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056388999999999996 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639452+0000", 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000132772 2024-09-21T22:50:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639526+0000", 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3773e-05 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639587+0000", 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1356e-05 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1189 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.184406149, 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085832630000000007, 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1189 2024-09-21T22:50:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7599999999999996e-07 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631141+0000", 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631184+0000", 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3043000000000001e-05 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631211+0000", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7149999999999999e-05 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.634622+0000", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034107759999999999 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.634884+0000", 2024-09-21T22:50:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026270800000000001 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.635491+0000", 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00060649200000000003 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639429+0000", 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039378099999999999 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639601+0000", 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000172566 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639665+0000", 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4215999999999996e-05 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639718+0000", 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2448999999999999e-05 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1190 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1842835840000001, 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018342312, 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1190 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631258+0000", 2024-09-21T22:50:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.13e-06 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631271+0000", 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2277e-05 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631296+0000", 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5466e-05 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631306+0000", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6399999999999992e-06 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.636550+0000", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005243826 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.636842+0000", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029216099999999998 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.637874+0000", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001031922 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644481+0000", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066069800000000001 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649278+0000", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047974660000000002 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649438+0000", 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159779 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649599+0000", 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016166499999999999 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1191 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1841499180000001, 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018524929999999998, 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1191 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631392+0000", 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.327e-06 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631403+0000", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1286e-05 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631417+0000", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3906999999999999e-05 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631426+0000", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7730000000000005e-06 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639898+0000", 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084718320000000003 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.640159+0000", 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026053799999999999 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.641256+0000", 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010972779999999999 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649043+0000", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007787459 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649654+0000", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061120499999999997 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649795+0000", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140073 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649916+0000", 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121252 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1192 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1840977920000002, 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018702169000000001, 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1192 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631444+0000", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8700000000000005e-07 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631448+0000", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4070000000000003e-06 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631485+0000", 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631496+0000", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1337e-05 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.642355+0000", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010859133 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.642573+0000", 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021829799999999999 2024-09-21T22:50:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.643635+0000", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010612989999999999 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649233+0000", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055979560000000003 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649944+0000", 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00071163800000000001 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650081+0000", 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013650199999999999 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650145+0000", 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4362999999999993e-05 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1193 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1840176859999998, 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018941726999999998, 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1193 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631524+0000", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1299999999999999e-07 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631534+0000", 2024-09-21T22:50:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0156999999999999e-05 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631546+0000", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.164e-05 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631554+0000", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0199999999999994e-06 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644532+0000", 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012978686999999999 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644662+0000", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012948899999999999 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.645392+0000", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072992399999999996 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650162+0000", 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047703590000000001 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650304+0000", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014169599999999999 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650393+0000", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8806000000000003e-05 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650465+0000", 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2136000000000002e-05 2024-09-21T22:50:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1194 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.183965003, 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01936705, 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1194 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3399999999999998e-07 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631581+0000", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4959999999999998e-06 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631606+0000", 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5089999999999999e-05 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631614+0000", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2300000000000008e-06 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.646403+0000", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014788875999999999 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.646673+0000", 2024-09-21T22:50:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026966700000000002 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.647843+0000", 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011701509999999999 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650628+0000", 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027844990000000002 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650668+0000", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0973000000000002e-05 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650830+0000", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000161769 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650943+0000", 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112565 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1195 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1839018729999999, 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027238346, 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1195 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631640+0000", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4e-07 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631650+0000", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0496000000000001e-05 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631672+0000", 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1250000000000002e-05 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631680+0000", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5969999999999992e-06 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.651088+0000", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019407569999999999 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.652841+0000", 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017528229999999999 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.653876+0000", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010356950000000001 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658659+0000", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047829939999999996 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658682+0000", 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2853e-05 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658761+0000", 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8988999999999999e-05 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658877+0000", 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011613899999999999 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1196 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1838485470000002, 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028067307, 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1196 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631693+0000", 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3300000000000001e-07 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631697+0000", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3599999999999998e-06 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631709+0000", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2064e-05 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631717+0000", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.593e-06 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.654952+0000", 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023235267 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.655198+0000", 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024591100000000002 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.655888+0000", 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068951500000000003 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658895+0000", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030075100000000001 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659588+0000", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00069257900000000002 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659694+0000", 2024-09-21T22:50:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106099 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659760+0000", 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5675999999999999e-05 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1197 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1838097340000004, 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028784578000000002, 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1197 2024-09-21T22:50:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631732+0000", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631736+0000", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3429999999999998e-06 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631759+0000", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2877000000000001e-05 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631767+0000", 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1729999999999995e-06 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.656584+0000", 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024816955000000002 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.656729+0000", 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014541200000000001 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.657707+0000", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097799600000000007 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659812+0000", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00210487 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660354+0000", 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054205599999999996 2024-09-21T22:50:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660446+0000", 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1935999999999995e-05 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660516+0000", 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9282999999999994e-05 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1803 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7616575890000004, 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0097822800000000008, 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1803 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053884+0000", 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053888+0000", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9369999999999997e-06 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053897+0000", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3400000000000004e-06 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053904+0000", 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0700000000000001e-06 2024-09-21T22:50:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058380+0000", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044756400000000003 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058513+0000", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013261499999999999 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058837+0000", 2024-09-21T22:50:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032426900000000002 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063168+0000", 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043308779999999998 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063526+0000", 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035849599999999998 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063605+0000", 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8891000000000001e-05 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063665+0000", 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0310999999999999e-05 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1880 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6138530429999998, 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083682879999999998, 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1880 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201689+0000", 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6400000000000001e-07 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201694+0000", 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9100000000000002e-06 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201709+0000", 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4348999999999999e-05 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201721+0000", 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.223e-05 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205069+0000", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033475139999999998 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205117+0000", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8128999999999999e-05 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205315+0000", 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019795300000000001 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.208298+0000", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029836590000000001 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.209953+0000", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016543969999999999 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210013+0000", 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0393999999999998e-05 2024-09-21T22:50:18.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210056+0000", 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2889000000000002e-05 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1881 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6138067070000002, 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084419609999999996, 2024-09-21T22:50:18.895 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1881 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201735+0000", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5199999999999996e-07 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201740+0000", 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8779999999999997e-06 2024-09-21T22:50:18.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201753+0000", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3026e-05 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201762+0000", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6420000000000002e-06 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205657+0000", 2024-09-21T22:50:18.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038946409999999999 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205747+0000", 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9629000000000005e-05 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205984+0000", 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000237505 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.209162+0000", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031778269999999998 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210096+0000", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093459700000000001 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210142+0000", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5720000000000003e-05 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210176+0000", 2024-09-21T22:50:18.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3844000000000003e-05 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2014 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.418023335, 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010643727, 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2014 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397519+0000", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0720000000000001e-06 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397524+0000", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1420000000000001e-06 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397528+0000", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7609999999999998e-06 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397532+0000", 2024-09-21T22:50:18.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0600000000000001e-06 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.399910+0000", 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023775060000000002 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.399961+0000", 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1323000000000003e-05 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400152+0000", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000190774 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407832+0000", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007679745 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408024+0000", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019234699999999999 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408098+0000", 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3428000000000002e-05 2024-09-21T22:50:18.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408161+0000", 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3569000000000001e-05 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2015 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4180037600000004, 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010750964999999999, 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2015 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8599999999999999e-07 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397539+0000", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.782e-06 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397543+0000", 2024-09-21T22:50:18.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4529999999999998e-06 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397545+0000", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6740000000000001e-06 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400728+0000", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031827919999999998 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400780+0000", 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1768000000000003e-05 2024-09-21T22:50:18.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401085+0000", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030565699999999998 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407841+0000", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067553020000000004 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408206+0000", 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000365115 2024-09-21T22:50:18.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408243+0000", 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6789000000000003e-05 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408288+0000", 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5346999999999998e-05 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2209 3.16 3:6bb22ccb:::rbd_data.111ea6ec99f1.000000000000010b:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1311409269999997, 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082901020000000006, 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2209 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684401+0000", 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0500000000000003e-07 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684405+0000", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3919999999999996e-06 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684415+0000", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6569999999999992e-06 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684420+0000", 2024-09-21T22:50:18.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5400000000000003e-06 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684488+0000", 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7451000000000002e-05 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684683+0000", 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019530700000000001 2024-09-21T22:50:18.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.686035+0000", 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013524850000000001 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692558+0000", 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0065231040000000001 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692583+0000", 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4189000000000002e-05 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692628+0000", 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5809000000000001e-05 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692690+0000", 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1463000000000004e-05 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2223 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1121569420000004, 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092793560000000008, 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2223 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.918 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.27e-07 2024-09-21T22:50:18.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703387+0000", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4549999999999998e-06 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703391+0000", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5770000000000004e-06 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703396+0000", 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7929999999999997e-06 2024-09-21T22:50:18.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707621+0000", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042251579999999997 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707719+0000", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8195000000000004e-05 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.708030+0000", 2024-09-21T22:50:18.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031057699999999998 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712552+0000", 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045221200000000001 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712573+0000", 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1135999999999999e-05 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712613+0000", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9835000000000001e-05 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712663+0000", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0182999999999997e-05 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2225 3.1c 3:3f3e9105:::rbd_data.111ea6ec99f1.0000000000000295:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:18.923 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1087073839999997, 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010523404, 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2225 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:18.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706835+0000", 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3430000000000001e-06 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706842+0000", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1080000000000004e-06 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706858+0000", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6350000000000001e-05 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706866+0000", 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706925+0000", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9172999999999997e-05 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707070+0000", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000144948 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.709041+0000", 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019702029999999998 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717189+0000", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081485010000000007 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717240+0000", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1069999999999997e-05 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717308+0000", 2024-09-21T22:50:18.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8011999999999999e-05 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717357+0000", 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619999999999999e-05 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3129 3.1d 3:bf2f0da1:::rbd_data.111ea6ec99f1.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.6318795050000001, 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098082800000000008, 2024-09-21T22:50:18.929 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3129 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183662+0000", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183671+0000", 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9959999999999999e-06 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183692+0000", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0662999999999999e-05 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183708+0000", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5444e-05 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.186410+0000", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027025320000000001 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:18.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.186651+0000", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024083199999999999 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.187201+0000", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055044200000000005 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.189161+0000", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019600149999999998 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193310+0000", 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041485610000000003 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193402+0000", 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2379000000000004e-05 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193470+0000", 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:18.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:18.935 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:18.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:18.937 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:18.930+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:717 read 0x10a0e0b thru 0x10a6302 (0x54f8 bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:719 punch from 0x281ddb to 0x28e4d4, (0xc6f9 bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:721 write 0xb5d648b thru 0xb5e1248 (0xadbe bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:722 read 0xa09488b thru 0xa0a1d40 (0xd4b6 bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:723 write 0xab213c2 thru 0xab27ff5 (0x6c34 bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:725 write 0xa4178ac thru 0xa42170e (0x9e63 bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:727 writesame 0x74753c3 thru 0x747fcad data_size 0x1(0xa8eb bytes) 2024-09-21T22:50:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:729 read 0x44c8480 thru 0x44d465f (0xc1e0 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:730 read 0x82a0fb8 thru 0x82aecf4 (0xdd3d bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:732 punch from 0x170fccc to 0x171764c, (0x7980 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:733 write 0x7e80058 thru 0x7e898e2 (0x988b bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:736 write 0x9ee0b9e thru 0x9ee8065 (0x74c8 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:737 write 0xa797e7f thru 0xa7a5060 (0xd1e2 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:739 write 0x21b8f49 thru 0x21bb430 (0x24e8 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:740 writesame 0x2377359 thru 0x2379358 data_size 0x20(0x2000 bytes) 2024-09-21T22:50:18.939 INFO:teuthology.orchestra.run.smithi094.stdout:742 writesame 0xb1395e4 thru 0xb13d4cf data_size 0x2(0x3eec bytes) 2024-09-21T22:50:18.942 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.942 INFO:teuthology.orchestra.run.smithi094.stdout:747 read 0x5fc034 thru 0x603852 (0x781f bytes) 2024-09-21T22:50:18.943 INFO:teuthology.orchestra.run.smithi094.stdout:748 read 0x382ee4e thru 0x38353b5 (0x6568 bytes) 2024-09-21T22:50:18.943 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:18.943 INFO:teuthology.orchestra.run.smithi094.stdout:750 write 0x50213c6 thru 0x5029db7 (0x89f2 bytes) 2024-09-21T22:50:18.948 INFO:teuthology.orchestra.run.smithi094.stdout:752 writesame 0x998f7ef thru 0x99945a5 data_size 0x17(0x4db7 bytes) 2024-09-21T22:50:18.956 INFO:teuthology.orchestra.run.smithi094.stdout:753 read 0x48021c thru 0x48a8bd (0xa6a2 bytes) 2024-09-21T22:50:18.958 INFO:teuthology.orchestra.run.smithi094.stdout:754 write 0xa3c0bc6 thru 0xa3cecda (0xe115 bytes) 2024-09-21T22:50:18.963 INFO:teuthology.orchestra.run.smithi094.stdout:755 read 0x75cac90 thru 0x75d4556 (0x98c7 bytes) 2024-09-21T22:50:18.963 INFO:teuthology.orchestra.run.smithi094.stdout:756 punch from 0x5724327 to 0x572c3b5, (0x808e bytes) 2024-09-21T22:50:18.964 INFO:teuthology.orchestra.run.smithi094.stdout:757 write 0x1ef9aa thru 0x1f5186 (0x57dd bytes) 2024-09-21T22:50:18.977 INFO:teuthology.orchestra.run.smithi094.stdout:758 writesame 0x469d1ab thru 0x46a6b91 data_size 0x17(0x99e7 bytes) 2024-09-21T22:50:18.985 INFO:teuthology.orchestra.run.smithi094.stdout:759 read 0x9c99c96 thru 0x9c9f3ce (0x5739 bytes) 2024-09-21T22:50:18.985 INFO:teuthology.orchestra.run.smithi094.stdout:760 write 0x74031b3 thru 0x740ca05 (0x9853 bytes) 2024-09-21T22:50:19.012 INFO:teuthology.orchestra.run.smithi094.stdout:763 write 0xec41b40 thru 0xec49a50 (0x7f11 bytes) 2024-09-21T22:50:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:765 read 0x601dfa thru 0x60cb25 (0xad2c bytes) 2024-09-21T22:50:19.032 INFO:teuthology.orchestra.run.smithi094.stdout:766 write 0xedbbacf thru 0xedc2e21 (0x7353 bytes) 2024-09-21T22:50:19.036 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:19.033+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:19.037 INFO:teuthology.orchestra.run.smithi094.stdout:767 write 0x5d5d16d thru 0x5d5ddab (0xc3f bytes) 2024-09-21T22:50:19.043 INFO:teuthology.orchestra.run.smithi094.stdout:769 punch from 0x54ca23d to 0x54cff91, (0x5d54 bytes) 2024-09-21T22:50:19.043 INFO:teuthology.orchestra.run.smithi094.stdout:770 read 0x63f6f5f thru 0x63faffe (0x40a0 bytes) 2024-09-21T22:50:19.043 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:19.043 INFO:teuthology.orchestra.run.smithi094.stdout:776 read 0x93209c5 thru 0x9328eab (0x84e7 bytes) 2024-09-21T22:50:19.043 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:19.050 INFO:teuthology.orchestra.run.smithi094.stdout:778 trunc from 0xedc2e22 to 0xc6bdedf 2024-09-21T22:50:19.069 INFO:teuthology.orchestra.run.smithi094.stdout:779 read 0x9e81c3 thru 0x9ec511 (0x434f bytes) 2024-09-21T22:50:19.069 INFO:teuthology.orchestra.run.smithi094.stdout:782 write 0x7fbfab2 thru 0x7fc35dd (0x3b2c bytes) 2024-09-21T22:50:19.075 INFO:teuthology.orchestra.run.smithi094.stdout:783 write 0x1caf122 thru 0x1cb513b (0x601a bytes) 2024-09-21T22:50:19.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:19.084 INFO:teuthology.orchestra.run.smithi094.stdout:784 write 0x43b6719 thru 0x43bb98e (0x5276 bytes) 2024-09-21T22:50:19.090 INFO:teuthology.orchestra.run.smithi094.stdout:786 write 0x4d392a3 thru 0x4d3f3ce (0x612c bytes) 2024-09-21T22:50:19.102 INFO:teuthology.orchestra.run.smithi094.stdout:787 read 0xb56f21 thru 0xb66255 (0xf335 bytes) 2024-09-21T22:50:19.102 INFO:teuthology.orchestra.run.smithi094.stdout:790 write 0x389908 thru 0x398bfc (0xf2f5 bytes) 2024-09-21T22:50:19.108 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:4847 3.11 3:8a21413e:::rbd_object_map.111e20a3ef21:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:19.106236+0000", 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003350207, 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00337354, 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:19.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4847 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106236+0000", 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106236+0000", 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106237+0000", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.09e-07 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106242+0000", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2739999999999998e-06 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106254+0000", 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2126999999999999e-05 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106260+0000", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2260000000000002e-06 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106329+0000", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8892000000000004e-05 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106400+0000", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0570999999999999e-05 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:19.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106653+0000", 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000252737 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.108928+0000", 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022751730000000001 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:4848 3.11 3:8a21413e:::rbd_object_map.111e20a3ef21:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:19.106290+0000", 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003296167, 2024-09-21T22:50:19.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003401173, 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4848 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106290+0000", 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106290+0000", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106291+0000", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106295+0000", 2024-09-21T22:50:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5340000000000001e-06 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106315+0000", 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9361e-05 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.106322+0000", 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3830000000000003e-06 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.107243+0000", 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092089600000000004 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.107434+0000", 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019111200000000001 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.108111+0000", 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00067709200000000002 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:50:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 2 2024-09-21T22:50:19.118 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:19.118 INFO:teuthology.orchestra.run.smithi094.stdout:791 read 0x2eadfdf thru 0x2eb3dab (0x5dcd bytes) 2024-09-21T22:50:19.118 INFO:teuthology.orchestra.run.smithi094.stdout:793 writesame 0x447f084 thru 0x4485d26 data_size 0x1d(0x6ca3 bytes) 2024-09-21T22:50:19.124 INFO:teuthology.orchestra.run.smithi094.stdout:794 read 0x10ed9af thru 0x10f06a5 (0x2cf7 bytes) 2024-09-21T22:50:19.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:19.131 INFO:teuthology.orchestra.run.smithi094.stdout:795 writesame 0xc83148d thru 0xc840da4 data_size 0x1(0xf918 bytes) 2024-09-21T22:50:19.137 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:19.133+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:19.149 INFO:teuthology.orchestra.run.smithi094.stdout:796 trunc from 0xc840da5 to 0x2281133 2024-09-21T22:50:19.186 INFO:teuthology.orchestra.run.smithi094.stdout:798 write 0x4840880 thru 0x48482b2 (0x7a33 bytes) 2024-09-21T22:50:19.191 INFO:teuthology.orchestra.run.smithi094.stdout:799 read 0x29bb031 thru 0x29c40f9 (0x90c9 bytes) 2024-09-21T22:50:19.191 INFO:teuthology.orchestra.run.smithi094.stdout:800 read 0x500f29 thru 0x50f491 (0xe569 bytes) 2024-09-21T22:50:19.211 INFO:teuthology.orchestra.run.smithi094.stdout:801 write 0xab08c83 thru 0xab0cd60 (0x40de bytes) 2024-09-21T22:50:19.215 INFO:teuthology.orchestra.run.smithi094.stdout:802 read 0x2e0ab7 thru 0x2e0d9e (0x2e8 bytes) 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:804 punch from 0x7818089 to 0x782507b, (0xcff2 bytes) 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:806 punch from 0x8740bf9 to 0x874e43b, (0xd842 bytes) 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:807 read 0x661d642 thru 0x662b13b (0xdafa bytes) 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:808 read 0x3130e8b thru 0x313bc8a (0xae00 bytes) 2024-09-21T22:50:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:810 write 0x9e52b8f thru 0x9e5d397 (0xa809 bytes) 2024-09-21T22:50:19.220 INFO:teuthology.orchestra.run.smithi094.stdout:813 writesame 0x7fa0f4a thru 0x7fa3c44 data_size 0x2f(0x2cfb bytes) 2024-09-21T22:50:19.223 INFO:teuthology.orchestra.run.smithi094.stdout:814 writesame 0x8e1df41 thru 0x8e25e36 data_size 0x3(0x7ef6 bytes) 2024-09-21T22:50:19.230 INFO:teuthology.orchestra.run.smithi094.stdout:815 punch from 0x87d72b8 to 0x87dac30, (0x3978 bytes) 2024-09-21T22:50:19.238 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:19.233+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:19.239 INFO:teuthology.orchestra.run.smithi094.stdout:816 write 0xc73bcf4 thru 0xc73cb07 (0xe14 bytes) 2024-09-21T22:50:19.243 INFO:teuthology.orchestra.run.smithi094.stdout:817 punch from 0x22b3405 to 0x22c3032, (0xfc2d bytes) 2024-09-21T22:50:19.243 INFO:teuthology.orchestra.run.smithi094.stdout:819 punch from 0x8a610b1 to 0x8a6fdbb, (0xed0a bytes) 2024-09-21T22:50:19.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:19.250 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:19.252 INFO:teuthology.orchestra.run.smithi094.stdout:821 writesame 0xed8c134 thru 0xed92f19 data_size 0x1b(0x6de6 bytes) 2024-09-21T22:50:19.252 INFO:teuthology.orchestra.run.smithi094.stdout:822 read 0x33f14f7 thru 0x33f6610 (0x511a bytes) 2024-09-21T22:50:19.252 INFO:teuthology.orchestra.run.smithi094.stdout:825 write 0x84bfc7b thru 0x84c4506 (0x488c bytes) 2024-09-21T22:50:19.254 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:19.254 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:19.254 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:19.254 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:19.254 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:19.257 INFO:teuthology.orchestra.run.smithi094.stdout:826 write 0x63510a1 thru 0x6357d89 (0x6ce9 bytes) 2024-09-21T22:50:19.260 INFO:teuthology.orchestra.run.smithi094.stdout:827 write 0x875d872 thru 0x8765e62 (0x85f1 bytes) 2024-09-21T22:50:19.266 INFO:teuthology.orchestra.run.smithi094.stdout:828 trunc from 0xed92f1a to 0xdc419bb 2024-09-21T22:50:19.268 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:19.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:19.273 INFO:teuthology.orchestra.run.smithi094.stdout:830 punch from 0x3f8ab6c to 0x3f90939, (0x5dcd bytes) 2024-09-21T22:50:19.273 INFO:teuthology.orchestra.run.smithi094.stdout:831 write 0x6aced49 thru 0x6ada465 (0xb71d bytes) 2024-09-21T22:50:19.278 INFO:teuthology.orchestra.run.smithi094.stdout:833 clone 5 order 23 su 1048576 sc 10 2024-09-21T22:50:19.285 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:19.285 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:19.301 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:19.302 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:19.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:19.318 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:19.335 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:19.335 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:19.336 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:19.339 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:19.333+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:19.354 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:19.354 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:19.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:19.358 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:19.384 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:19.385 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:19.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:19.440 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:19.438+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:19.523 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:19.523 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:19.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:19.585 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:19.581+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:19.642 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:19.637+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:19.661 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:19.661 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:19.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:19.743 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:19.738+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:19.824 INFO:teuthology.orchestra.run.smithi099.stdout:get_pg_num_history pg_num_history pg_num_history(e23 pg_nums {1={17=1},2={20=8}} deleted_pools ) 2024-09-21T22:50:19.824 INFO:teuthology.orchestra.run.smithi099.stdout:Importing pgid 2.7 2024-09-21T22:50:19.829 INFO:teuthology.orchestra.run.smithi099.stdout:write_pg epoch 23 info 2.7( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-21T22:50:19.829 INFO:teuthology.orchestra.run.smithi099.stdout:Import successful 2024-09-21T22:50:19.841 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:19.841 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:19.843 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:19.837+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:19.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:19.945 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:19.941+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:20.025 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.025 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:20.081 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:20.077+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:20.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:50:20.173 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:20.169+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:20.212 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.212 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:20.236 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone4 from 0xdc419bb (overlap 0x57498c) to 0x345f54 2024-09-21T22:50:20.246 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:20.242+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:20.347 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:20.342+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:20.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /var/log/ceph/exp.2.7.6 2024-09-21T22:50:20.371 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.371 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:20.398 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T22:50:20.400 INFO:teuthology.orchestra.run.smithi094.stdout:839 punch from 0xa6adc27 to 0xa6b1659, (0x3a32 bytes) 2024-09-21T22:50:20.405 INFO:teuthology.orchestra.run.smithi094.stdout:840 writesame 0x28a84d2 thru 0x28a916e data_size 0x1(0xc9d bytes) 2024-09-21T22:50:20.420 INFO:teuthology.orchestra.run.smithi094.stdout:841 write 0xcf623c4 thru 0xcf68f71 (0x6bae bytes) 2024-09-21T22:50:20.426 INFO:teuthology.orchestra.run.smithi094.stdout:842 writesame 0x1be502a thru 0x1beda3c data_size 0x1(0x8a13 bytes) 2024-09-21T22:50:20.447 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:20.441+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:20.461 INFO:teuthology.orchestra.run.smithi094.stdout:844 punch from 0x463555d to 0x4637290, (0x1d33 bytes) 2024-09-21T22:50:20.461 INFO:teuthology.orchestra.run.smithi094.stdout:848 punch from 0xafd29b3 to 0xafd5347, (0x2994 bytes) 2024-09-21T22:50:20.461 INFO:teuthology.orchestra.run.smithi094.stdout:849 write 0xbd3e115 thru 0xbd446b5 (0x65a1 bytes) 2024-09-21T22:50:20.471 INFO:teuthology.orchestra.run.smithi094.stdout:850 trunc from 0xdc419bb to 0xa97bbd6 2024-09-21T22:50:20.495 INFO:teuthology.orchestra.run.smithi094.stdout:851 writesame 0x59aa5a8 thru 0x59ab5a2 data_size 0x1(0xffb bytes) 2024-09-21T22:50:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:853 punch from 0x23aae32 to 0x23ba97c, (0xfb4a bytes) 2024-09-21T22:50:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:854 read 0xa3795cc thru 0xa382ccc (0x9701 bytes) 2024-09-21T22:50:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:855 punch from 0x69d92b7 to 0x69da89a, (0x15e3 bytes) 2024-09-21T22:50:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:860 read 0x9bdaf57 thru 0x9be2ab1 (0x7b5b bytes) 2024-09-21T22:50:20.509 INFO:teuthology.orchestra.run.smithi094.stdout:861 trunc from 0xa97bbd6 to 0x7b9bfbc 2024-09-21T22:50:20.511 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.511 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:20.548 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:20.546+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:20.561 INFO:teuthology.orchestra.run.smithi094.stdout:862 punch from 0x2ddc5d3 to 0x2de8224, (0xbc51 bytes) 2024-09-21T22:50:20.572 INFO:teuthology.orchestra.run.smithi094.stdout:863 writesame 0x9090dab thru 0x909fca9 data_size 0x3b(0xeeff bytes) 2024-09-21T22:50:20.591 INFO:teuthology.orchestra.run.smithi094.stdout:865 write 0xdba817a thru 0xdbb018a (0x8011 bytes) 2024-09-21T22:50:20.616 INFO:teuthology.orchestra.run.smithi094.stdout:866 trunc from 0xdbb018b to 0xca814d 2024-09-21T22:50:20.649 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:20.645+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:20.669 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.669 INFO:teuthology.orchestra.run.smithi094.stdout:868 read 0x9a07b3 thru 0x9acdce (0xc61c bytes) 2024-09-21T22:50:20.685 INFO:teuthology.orchestra.run.smithi094.stdout:869 write 0x3766a0e thru 0x37696f5 (0x2ce8 bytes) 2024-09-21T22:50:20.688 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.692 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.692 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.708 INFO:teuthology.orchestra.run.smithi094.stdout:872 write 0x9754f38 thru 0x975fae1 (0xabaa bytes) 2024-09-21T22:50:20.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:20.735 INFO:teuthology.orchestra.run.smithi094.stdout:874 writesame 0xb9cfd94 thru 0xb9cfdec data_size 0x1(0x59 bytes) 2024-09-21T22:50:20.736 INFO:teuthology.orchestra.run.smithi094.stdout:876 writesame 0x2dda7d9 thru 0x2ddc1a0 data_size 0xf(0x19c8 bytes) 2024-09-21T22:50:20.739 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.739 INFO:teuthology.orchestra.run.smithi094.stdout:878 read 0x556e7ed thru 0x557b717 (0xcf2b bytes) 2024-09-21T22:50:20.750 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:20.745+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:20.756 INFO:teuthology.orchestra.run.smithi094.stdout:879 writesame 0xee3cefe thru 0xee430b3 data_size 0x16(0x61b6 bytes) 2024-09-21T22:50:20.756 INFO:teuthology.orchestra.run.smithi094.stdout:880 writesame 0x64743af thru 0x6476e94 data_size 0x22(0x2ae6 bytes) 2024-09-21T22:50:20.764 INFO:teuthology.orchestra.run.smithi094.stdout:881 punch from 0x310e572 to 0x3118651, (0xa0df bytes) 2024-09-21T22:50:20.764 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.781 INFO:teuthology.orchestra.run.smithi094.stdout:883 trunc from 0xee430b4 to 0x5d1e496 2024-09-21T22:50:20.793 INFO:teuthology.orchestra.run.smithi094.stdout:884 read 0x40013ff thru 0x4010012 (0xec14 bytes) 2024-09-21T22:50:20.794 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.794 INFO:teuthology.orchestra.run.smithi094.stdout:887 read 0x43f8eaa thru 0x43fe7ec (0x5943 bytes) 2024-09-21T22:50:20.795 INFO:teuthology.orchestra.run.smithi094.stdout:889 read 0x5bc5826 thru 0x5bcfa69 (0xa244 bytes) 2024-09-21T22:50:20.795 INFO:teuthology.orchestra.run.smithi094.stdout:891 read 0x4e51f90 thru 0x4e5aa3f (0x8ab0 bytes) 2024-09-21T22:50:20.796 INFO:teuthology.orchestra.run.smithi094.stdout:892 punch from 0x2b6f075 to 0x2b77e7a, (0x8e05 bytes) 2024-09-21T22:50:20.796 INFO:teuthology.orchestra.run.smithi094.stdout:893 writesame 0xa68ab0 thru 0xa77c3e data_size 0x3(0xf18f bytes) 2024-09-21T22:50:20.807 INFO:teuthology.orchestra.run.smithi094.stdout:894 punch from 0x5ba05f1 to 0x5ba878b, (0x819a bytes) 2024-09-21T22:50:20.813 INFO:teuthology.orchestra.run.smithi094.stdout:895 trunc from 0x5d1e496 to 0x9ef97d5 2024-09-21T22:50:20.824 INFO:teuthology.orchestra.run.smithi094.stdout:896 writesame 0x4844680 thru 0x484d633 data_size 0x22(0x8fb4 bytes) 2024-09-21T22:50:20.828 INFO:teuthology.orchestra.run.smithi094.stdout:898 read 0x26fc1f6 thru 0x27099fc (0xd807 bytes) 2024-09-21T22:50:20.828 INFO:teuthology.orchestra.run.smithi094.stdout:900 read 0x86f8fb8 thru 0x86ff4eb (0x6534 bytes) 2024-09-21T22:50:20.829 INFO:teuthology.orchestra.run.smithi094.stdout:901 read 0x1962631 thru 0x196d692 (0xb062 bytes) 2024-09-21T22:50:20.830 INFO:teuthology.orchestra.run.smithi094.stdout:902 writesame 0x2b27488 thru 0x2b3718c data_size 0x9(0xfd05 bytes) 2024-09-21T22:50:20.851 INFO:teuthology.orchestra.run.smithi094.stdout:904 writesame 0xc9a02cb thru 0xc9aea40 data_size 0xd(0xe776 bytes) 2024-09-21T22:50:20.851 INFO:teuthology.orchestra.run.smithi094.stdout:905 writesame 0x3ebd2bc thru 0x3ecb473 data_size 0x3e(0xe1b8 bytes) 2024-09-21T22:50:20.851 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:20.845+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:20.855 INFO:teuthology.orchestra.run.smithi094.stdout:906 read 0x1ec3950 thru 0x1ec90e6 (0x5797 bytes) 2024-09-21T22:50:20.855 INFO:teuthology.orchestra.run.smithi094.stdout:907 write 0x2c72ab8 thru 0x2c75ca1 (0x31ea bytes) 2024-09-21T22:50:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:908 write 0x755cb7d thru 0x755e947 (0x1dcb bytes) 2024-09-21T22:50:20.867 INFO:teuthology.orchestra.run.smithi094.stdout:909 read 0x941b4c1 thru 0x9429266 (0xdda6 bytes) 2024-09-21T22:50:20.868 INFO:teuthology.orchestra.run.smithi094.stdout:910 writesame 0x9b8a1 thru 0xa2063 data_size 0x1(0x67c3 bytes) 2024-09-21T22:50:20.874 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:20.874 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:20.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:20.895 INFO:teuthology.orchestra.run.smithi094.stdout:912 write 0x928ed52 thru 0x9291a11 (0x2cc0 bytes) 2024-09-21T22:50:20.901 INFO:teuthology.orchestra.run.smithi094.stdout:914 write 0x90f8fb1 thru 0x9105b11 (0xcb61 bytes) 2024-09-21T22:50:20.911 INFO:teuthology.orchestra.run.smithi094.stdout:915 read 0xbb9177c thru 0xbba12c3 (0xfb48 bytes) 2024-09-21T22:50:20.911 INFO:teuthology.orchestra.run.smithi094.stdout:916 write 0x3df103a thru 0x3df63c0 (0x5387 bytes) 2024-09-21T22:50:20.924 INFO:teuthology.orchestra.run.smithi094.stdout:917 writesame 0xdd35634 thru 0xdd367ea data_size 0x1(0x11b7 bytes) 2024-09-21T22:50:20.924 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.924 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.924 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.925 INFO:teuthology.orchestra.run.smithi094.stdout:925 writesame 0x15557b thru 0x15c853 data_size 0x1(0x72d9 bytes) 2024-09-21T22:50:20.943 INFO:teuthology.orchestra.run.smithi094.stdout:927 read 0xa453848 thru 0xa45dc46 (0xa3ff bytes) 2024-09-21T22:50:20.943 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.943 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.943 INFO:teuthology.orchestra.run.smithi094.stdout:931 write 0xb9a1c1c thru 0xb9aa661 (0x8a46 bytes) 2024-09-21T22:50:20.951 INFO:teuthology.orchestra.run.smithi094.stdout:934 punch from 0xbdc44 to 0xc961e, (0xb9da bytes) 2024-09-21T22:50:20.951 INFO:teuthology.orchestra.run.smithi094.stdout:935 write 0x6323973 thru 0x6325483 (0x1b11 bytes) 2024-09-21T22:50:20.951 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:20.945+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:20.959 INFO:teuthology.orchestra.run.smithi094.stdout:936 write 0xbeb223c thru 0xbeba6f4 (0x84b9 bytes) 2024-09-21T22:50:20.967 INFO:teuthology.orchestra.run.smithi094.stdout:937 punch from 0x66141cc to 0x6615c95, (0x1ac9 bytes) 2024-09-21T22:50:20.967 INFO:teuthology.orchestra.run.smithi094.stdout:938 write 0x5f5d975 thru 0x5f5eb6f (0x11fb bytes) 2024-09-21T22:50:20.971 INFO:teuthology.orchestra.run.smithi094.stdout:941 read 0x8a2e58c thru 0x8a3567b (0x70f0 bytes) 2024-09-21T22:50:20.971 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:20.971 INFO:teuthology.orchestra.run.smithi094.stdout:944 punch from 0xdc8fb6a to 0xdc92477, (0x290d bytes) 2024-09-21T22:50:20.971 INFO:teuthology.orchestra.run.smithi094.stdout:945 write 0x5274a40 thru 0x52784f0 (0x3ab1 bytes) 2024-09-21T22:50:20.977 INFO:teuthology.orchestra.run.smithi094.stdout:947 read 0x532732 thru 0x539cef (0x75be bytes) 2024-09-21T22:50:20.978 INFO:teuthology.orchestra.run.smithi094.stdout:948 read 0x309550d thru 0x30a4ea3 (0xf997 bytes) 2024-09-21T22:50:21.004 INFO:teuthology.orchestra.run.smithi094.stdout:949 trunc from 0xdd367eb to 0x301965c 2024-09-21T22:50:21.028 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:21.028 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:21.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:21.054 INFO:teuthology.orchestra.run.smithi094.stdout:950 writesame 0x6f80aae thru 0x6f85750 data_size 0x1(0x4ca3 bytes) 2024-09-21T22:50:21.055 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:21.049+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:21.074 INFO:teuthology.orchestra.run.smithi094.stdout:951 write 0xdb3aeb8 thru 0xdb418f9 (0x6a42 bytes) 2024-09-21T22:50:21.081 INFO:teuthology.orchestra.run.smithi094.stdout:952 writesame 0x68e8533 thru 0x68ec8c3 data_size 0x7(0x4391 bytes) 2024-09-21T22:50:21.087 INFO:teuthology.orchestra.run.smithi094.stdout:953 writesame 0x73a5ea7 thru 0x73b4f86 data_size 0xa4(0xf0e0 bytes) 2024-09-21T22:50:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:50:21.105 INFO:teuthology.orchestra.run.smithi094.stdout:954 punch from 0x6e2fe19 to 0x6e397c7, (0x99ae bytes) 2024-09-21T22:50:21.115 INFO:teuthology.orchestra.run.smithi094.stdout:955 write 0xe9021e0 thru 0xe908da7 (0x6bc8 bytes) 2024-09-21T22:50:21.122 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.123 INFO:teuthology.orchestra.run.smithi094.stdout:959 write 0x6048915 thru 0x60549c5 (0xc0b1 bytes) 2024-09-21T22:50:21.131 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.132 INFO:teuthology.orchestra.run.smithi094.stdout:962 read 0x5dca8de thru 0x5dcc2d3 (0x19f6 bytes) 2024-09-21T22:50:21.132 INFO:teuthology.orchestra.run.smithi094.stdout:964 write 0xc4be4d1 thru 0xc4c115b (0x2c8b bytes) 2024-09-21T22:50:21.140 INFO:teuthology.orchestra.run.smithi094.stdout:966 write 0x70ddafd thru 0x70e247c (0x4980 bytes) 2024-09-21T22:50:21.147 INFO:teuthology.orchestra.run.smithi094.stdout:968 writesame 0xe14de3f thru 0xe15b78f data_size 0x1(0xd951 bytes) 2024-09-21T22:50:21.152 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:21.150+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:21.183 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:21.183 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:21.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:21.218 INFO:teuthology.orchestra.run.smithi094.stdout:969 writesame 0xbc6c024 thru 0xbc7ae69 data_size 0x7(0xee46 bytes) 2024-09-21T22:50:21.235 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.235 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.235 INFO:teuthology.orchestra.run.smithi094.stdout:974 write 0x6135317 thru 0x613ed06 (0x99f0 bytes) 2024-09-21T22:50:21.241 INFO:teuthology.orchestra.run.smithi094.stdout:975 punch from 0x4ce4855 to 0x4ce8258, (0x3a03 bytes) 2024-09-21T22:50:21.241 INFO:teuthology.orchestra.run.smithi094.stdout:977 read 0x3c1aa02 thru 0x3c1d5e5 (0x2be4 bytes) 2024-09-21T22:50:21.242 INFO:teuthology.orchestra.run.smithi094.stdout:980 writesame 0x7ab6187 thru 0x7ac101e data_size 0x94(0xae98 bytes) 2024-09-21T22:50:21.245 INFO:teuthology.orchestra.run.smithi094.stdout:984 punch from 0x2a65a7 to 0x2b3918, (0xd371 bytes) 2024-09-21T22:50:21.254 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:21.250+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:21.255 INFO:teuthology.orchestra.run.smithi094.stdout:988 trunc from 0xe908da8 to 0x8f2a0ff 2024-09-21T22:50:21.279 INFO:teuthology.orchestra.run.smithi094.stdout:989 read 0x2c19cdc thru 0x2c278e6 (0xdc0b bytes) 2024-09-21T22:50:21.295 INFO:teuthology.orchestra.run.smithi094.stdout:991 write 0xb35c6ee thru 0xb36bb21 (0xf434 bytes) 2024-09-21T22:50:21.304 INFO:teuthology.orchestra.run.smithi094.stdout:995 writesame 0x8cdc03e thru 0x8ce02ee data_size 0x15(0x42b1 bytes) 2024-09-21T22:50:21.311 INFO:teuthology.orchestra.run.smithi094.stdout:996 read 0xa68450a thru 0xa684f5a (0xa51 bytes) 2024-09-21T22:50:21.311 INFO:teuthology.orchestra.run.smithi094.stdout:998 read 0x79fb404 thru 0x79fe649 (0x3246 bytes) 2024-09-21T22:50:21.311 INFO:teuthology.orchestra.run.smithi094.stdout:999 writesame 0x45b96ad thru 0x45c8b99 data_size 0x1(0xf4ed bytes) 2024-09-21T22:50:21.349 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:21.349 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:21.354 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:21.349+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:21.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:21.363 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:21.391 INFO:teuthology.orchestra.run.smithi094.stdout:1000 trunc from 0xb36bb22 to 0x60ffa45 2024-09-21T22:50:21.410 INFO:teuthology.orchestra.run.smithi094.stdout:1001 read 0x99b66e thru 0x9a029a (0x4c2d bytes) 2024-09-21T22:50:21.410 INFO:teuthology.orchestra.run.smithi094.stdout:1003 read 0x4e9f44f thru 0x4ea267a (0x322c bytes) 2024-09-21T22:50:21.411 INFO:teuthology.orchestra.run.smithi094.stdout:1004 read 0x24f5948 thru 0x24ff213 (0x98cc bytes) 2024-09-21T22:50:21.429 INFO:teuthology.orchestra.run.smithi094.stdout:1005 write 0xd052741 thru 0xd0572b1 (0x4b71 bytes) 2024-09-21T22:50:21.435 INFO:teuthology.orchestra.run.smithi094.stdout:1006 write 0xac7c87a thru 0xac888eb (0xc072 bytes) 2024-09-21T22:50:21.440 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.440 INFO:teuthology.orchestra.run.smithi094.stdout:1009 punch from 0x1616118 to 0x1618ca1, (0x2b89 bytes) 2024-09-21T22:50:21.452 INFO:teuthology.orchestra.run.smithi094.stdout:1010 trunc from 0xd0572b2 to 0x4620ecc 2024-09-21T22:50:21.454 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:21.449+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:21.478 INFO:teuthology.orchestra.run.smithi094.stdout:1011 read 0xfe2e51 thru 0xfea1bb (0x736b bytes) 2024-09-21T22:50:21.488 INFO:teuthology.orchestra.run.smithi094.stdout:1012 writesame 0x55594c6 thru 0x5559a34 data_size 0x1(0x56f bytes) 2024-09-21T22:50:21.500 INFO:teuthology.orchestra.run.smithi094.stdout:1013 write 0x8dee457 thru 0x8df9930 (0xb4da bytes) 2024-09-21T22:50:21.502 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:21.502 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:1014 trunc from 0x8df9931 to 0x3ccb37e 2024-09-21T22:50:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:21.516 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:21.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:21.530 INFO:teuthology.orchestra.run.smithi094.stdout:1015 punch from 0x3b1688a to 0x3b2314a, (0xc8c0 bytes) 2024-09-21T22:50:21.535 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:21.535 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:21.549 INFO:teuthology.orchestra.run.smithi094.stdout:1016 writesame 0xc7f182e thru 0xc7f61ed data_size 0x50(0x49c0 bytes) 2024-09-21T22:50:21.549 INFO:teuthology.orchestra.run.smithi094.stdout:1017 punch from 0x54f8d4a to 0x54f99b8, (0xc6e bytes) 2024-09-21T22:50:21.549 INFO:teuthology.orchestra.run.smithi094.stdout:1018 read 0x8f84f0b thru 0x8f84f39 (0x2f bytes) 2024-09-21T22:50:21.549 INFO:teuthology.orchestra.run.smithi094.stdout:1019 read 0x465b724 thru 0x465fa8c (0x4369 bytes) 2024-09-21T22:50:21.549 INFO:teuthology.orchestra.run.smithi094.stdout:1020 write 0xb4df140 thru 0xb4edc03 (0xeac4 bytes) 2024-09-21T22:50:21.552 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:21.553 INFO:teuthology.orchestra.run.smithi094.stdout:1021 write 0x9d40849 thru 0x9d4587d (0x5035 bytes) 2024-09-21T22:50:21.556 INFO:teuthology.orchestra.run.smithi094.stdout:1023 write 0x103489 thru 0x10543a (0x1fb2 bytes) 2024-09-21T22:50:21.558 INFO:teuthology.orchestra.run.smithi094.stdout:1024 trunc from 0xc7f61ee to 0xc854b1e 2024-09-21T22:50:21.560 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:21.558+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:21.565 INFO:teuthology.orchestra.run.smithi094.stdout:1025 read 0x23087e0 thru 0x2311205 (0x8a26 bytes) 2024-09-21T22:50:21.566 INFO:teuthology.orchestra.run.smithi094.stdout:1026 read 0x5ce98a7 thru 0x5cea352 (0xaac bytes) 2024-09-21T22:50:21.566 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.567 INFO:teuthology.orchestra.run.smithi094.stdout:1029 trunc from 0xc854b1e to 0xbbe23de 2024-09-21T22:50:21.570 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:21.570 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:21.574 INFO:teuthology.orchestra.run.smithi094.stdout:1030 write 0x5b3c326 thru 0x5b3fd06 (0x39e1 bytes) 2024-09-21T22:50:21.578 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:1032 read 0x9969445 thru 0x997613b (0xccf7 bytes) 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:1033 read 0x8221c9e thru 0x822f6ac (0xda0f bytes) 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:1034 read 0xb40f3f thru 0xb44ddd (0x3e9f bytes) 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:1036 read 0xa687069 thru 0xa68e60e (0x75a6 bytes) 2024-09-21T22:50:21.579 INFO:teuthology.orchestra.run.smithi094.stdout:1037 read 0x6e0ef0e thru 0x6e135ad (0x46a0 bytes) 2024-09-21T22:50:21.580 INFO:teuthology.orchestra.run.smithi094.stdout:1038 read 0x136f6ad thru 0x1376e98 (0x77ec bytes) 2024-09-21T22:50:21.580 INFO:teuthology.orchestra.run.smithi094.stdout:1039 write 0x26f1b66 thru 0x26ff6ab (0xdb46 bytes) 2024-09-21T22:50:21.581 INFO:teuthology.orchestra.run.smithi094.stdout:1040 write 0x11ede46 thru 0x11fd15a (0xf315 bytes) 2024-09-21T22:50:21.588 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:21.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:21.593 INFO:teuthology.orchestra.run.smithi094.stdout:1042 write 0xe63fc82 thru 0xe64630e (0x668d bytes) 2024-09-21T22:50:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:1043 read 0x89d77f2 thru 0x89e676f (0xef7e bytes) 2024-09-21T22:50:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:1045 writesame 0xc3c59b7 thru 0xc3c8dfb data_size 0x1(0x3445 bytes) 2024-09-21T22:50:21.605 INFO:teuthology.orchestra.run.smithi094.stdout:1046 read 0x626227b thru 0x6264d7b (0x2b01 bytes) 2024-09-21T22:50:21.607 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:21.607 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:21.619 INFO:teuthology.orchestra.run.smithi094.stdout:1047 trunc from 0xe64630f to 0x607f0c7 2024-09-21T22:50:21.633 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:21.635 INFO:teuthology.orchestra.run.smithi094.stdout:1048 punch from 0x41db4a7 to 0x41db6aa, (0x203 bytes) 2024-09-21T22:50:21.635 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.635 INFO:teuthology.orchestra.run.smithi094.stdout:1050 writesame 0x56f5be9 thru 0x56fb677 data_size 0x1(0x5a8f bytes) 2024-09-21T22:50:21.636 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:21.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:21.649 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:21.649 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:21.657 INFO:teuthology.orchestra.run.smithi094.stdout:1054 trunc from 0x607f0c7 to 0xf3580f 2024-09-21T22:50:21.657 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:21.653+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:21.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:21.675 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.692 INFO:teuthology.orchestra.run.smithi094.stdout:1057 write 0x4e11caf thru 0x4e14f2e (0x3280 bytes) 2024-09-21T22:50:21.695 INFO:teuthology.orchestra.run.smithi094.stdout:1060 read 0x1835866 thru 0x183ad30 (0x54cb bytes) 2024-09-21T22:50:21.695 INFO:teuthology.orchestra.run.smithi094.stdout:1063 read 0x16d651a thru 0x16e1260 (0xad47 bytes) 2024-09-21T22:50:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:1066 writesame 0x677193a thru 0x677996d data_size 0x1e(0x8034 bytes) 2024-09-21T22:50:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:1067 read 0x23fea11 thru 0x2405faf (0x759f bytes) 2024-09-21T22:50:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:1068 read 0x3c3c85f thru 0x3c427ef (0x5f91 bytes) 2024-09-21T22:50:21.711 INFO:teuthology.orchestra.run.smithi094.stdout:1069 read 0x25414cd thru 0x2541e55 (0x989 bytes) 2024-09-21T22:50:21.712 INFO:teuthology.orchestra.run.smithi094.stdout:1070 read 0x1c90fcc thru 0x1c9eeb6 (0xdeeb bytes) 2024-09-21T22:50:21.713 INFO:teuthology.orchestra.run.smithi094.stdout:1071 punch from 0x651e64b to 0x652289d, (0x4252 bytes) 2024-09-21T22:50:21.735 INFO:teuthology.orchestra.run.smithi094.stdout:1072 writesame 0xa9d240d thru 0xa9e05a4 data_size 0x4(0xe198 bytes) 2024-09-21T22:50:21.735 INFO:teuthology.orchestra.run.smithi094.stdout:1073 writesame 0x3918bc0 thru 0x391d46a data_size 0x35(0x48ab bytes) 2024-09-21T22:50:21.737 INFO:teuthology.orchestra.run.smithi094.stdout:1074 punch from 0x98a99d8 to 0x98ae5fd, (0x4c25 bytes) 2024-09-21T22:50:21.737 INFO:teuthology.orchestra.run.smithi094.stdout:1075 write 0x4d2219f thru 0x4d249bb (0x281d bytes) 2024-09-21T22:50:21.739 INFO:teuthology.orchestra.run.smithi094.stdout:1077 write 0x3748964 thru 0x3758165 (0xf802 bytes) 2024-09-21T22:50:21.757 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:21.754+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:21.761 INFO:teuthology.orchestra.run.smithi094.stdout:1078 write 0xcb01a27 thru 0xcb0ea76 (0xd050 bytes) 2024-09-21T22:50:21.766 INFO:teuthology.orchestra.run.smithi094.stdout:1079 write 0x381f978 thru 0x38295f0 (0x9c79 bytes) 2024-09-21T22:50:21.769 INFO:teuthology.orchestra.run.smithi094.stdout:1080 write 0xab4f265 thru 0xab5a42d (0xb1c9 bytes) 2024-09-21T22:50:21.804 INFO:teuthology.orchestra.run.smithi094.stdout:1082 trunc from 0xcb0ea77 to 0xd4e35e 2024-09-21T22:50:21.831 INFO:teuthology.orchestra.run.smithi094.stdout:1084 trunc from 0xd4e35e to 0x5b25e7 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:451 3.1a 3:58b09410:::rbd_data.111ecbf88771.0000000000000072:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3620299639999995, 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074162789999999996, 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 451 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469993+0000", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.469994+0000", 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.626e-06 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470004+0000", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4159999999999993e-06 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470020+0000", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6447e-05 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470031+0000", 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470093+0000", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2024000000000001e-05 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470206+0000", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113238 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.471324+0000", 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001117364 2024-09-21T22:50:21.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477262+0000", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059383550000000002 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477344+0000", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1831999999999998e-05 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477378+0000", 2024-09-21T22:50:21.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3618000000000002e-05 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477409+0000", 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1263000000000002e-05 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:452 3.a 3:56d9bfad:::rbd_data.111ecbf88771.0000000000000073:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3619159399999994, 2024-09-21T22:50:21.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072291539999999998, 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 452 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470107+0000", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470108+0000", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.08e-06 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470114+0000", 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:21.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470124+0000", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0465999999999999e-05 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470132+0000", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0670000000000003e-06 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470202+0000", 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0653000000000003e-05 2024-09-21T22:50:21.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.470412+0000", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209817 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477222+0000", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068098330000000004 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477263+0000", 2024-09-21T22:50:21.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0694e-05 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.477336+0000", 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3250999999999994e-05 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:653 3.a 3:551485c8:::rbd_data.111ecbf88771.00000000000000fa:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.1103430920000008, 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082106869999999995, 2024-09-21T22:50:21.847 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 653 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721679+0000", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721681+0000", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5689999999999999e-06 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721689+0000", 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-21T22:50:21.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721709+0000", 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9343000000000001e-05 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721718+0000", 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4150000000000005e-06 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721783+0000", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535000000000005e-05 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.721876+0000", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3410000000000002e-05 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729625+0000", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077490689999999999 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729751+0000", 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126062 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:12.729890+0000", 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013910300000000001 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1008 3.14 3:2c7bff05:::rbd_data.111ecbf88771.0000000000000030:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:21.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.6730919530000001, 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068041639999999997, 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1008 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:21.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158931+0000", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158933+0000", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158945+0000", 2024-09-21T22:50:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.158980+0000", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4980000000000001e-05 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159000+0000", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0015999999999999e-05 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159308+0000", 2024-09-21T22:50:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030743099999999999 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.159539+0000", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023104500000000001 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.160019+0000", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048036299999999998 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.162010+0000", 2024-09-21T22:50:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019912760000000002 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165428+0000", 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034175820000000002 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165584+0000", 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015635900000000001 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.165735+0000", 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015054900000000001 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:1033 3.1b e29/24 3:dd3d94e9:::rbd_data.111ecbf88771.0000000000000074:head v 29'9, mlcod=28'6)", 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.6586175940000008, 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070721250000000003, 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173405+0000", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173407+0000", 2024-09-21T22:50:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6759999999999999e-06 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173420+0000", 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3224000000000001e-05 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173446+0000", 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6118999999999998e-05 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173465+0000", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8542999999999998e-05 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173677+0000", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000212936 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.173731+0000", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3981999999999999e-05 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180381+0000", 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066499920000000004 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.180477+0000", 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5653000000000001e-05 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:2014 3.1d e29/24 3:bb619233:::rbd_object_map.111ea6ec99f1:head v 29'189, mlcod=29'184)", 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.431780002, 2024-09-21T22:50:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076346629999999999, 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400243+0000", 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4899999999999995e-07 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400249+0000", 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6119999999999998e-06 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400262+0000", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2965e-05 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400269+0000", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0360000000000001e-06 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400349+0000", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0105999999999995e-05 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400368+0000", 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8749e-05 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407812+0000", 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074442179999999998 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407877+0000", 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5427999999999998e-05 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3027 3.3 3:c2ee4b09:::rbd_header.111ea6ec99f1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7508413149999997, 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071998269999999998, 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3027 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081181+0000", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081185+0000", 2024-09-21T22:50:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1729999999999997e-06 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081204+0000", 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8940000000000002e-05 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081244+0000", 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0092000000000002e-05 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081264+0000", 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9344e-05 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081411+0000", 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014733500000000001 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.081421+0000", 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6250000000000002e-06 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.085977+0000", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045564969999999996 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.086051+0000", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3627999999999994e-05 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.086406+0000", 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035498300000000001 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.087041+0000", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063464299999999995 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088313+0000", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001272872 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088350+0000", 2024-09-21T22:50:21.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6795999999999998e-05 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.088381+0000", 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0898999999999997e-05 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3128 3.13 e31/24 3:cd5340d6:::rbd_data.111ea6ec99f1.0000000000000005:head v 31'39, mlcod=30'36)", 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6471542589999997, 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077291139999999996, 2024-09-21T22:50:21.873 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184868+0000", 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184870+0000", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7680000000000001e-06 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184884+0000", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4115e-05 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184911+0000", 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6797000000000001e-05 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.184924+0000", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2828e-05 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185057+0000", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133287 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185105+0000", 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7573e-05 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192499+0000", 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073943059999999998 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192597+0000", 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999999e-05 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.877 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3125 3.11 e31/24 3:8b63553f:::rbd_data.111ea6ec99f1.0000000000000002:head v 31'40, mlcod=30'37)", 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6469815380000004, 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075503890000000002, 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185041+0000", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185042+0000", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185050+0000", 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8279999999999992e-06 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185071+0000", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0726000000000001e-05 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185089+0000", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8280000000000001e-05 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185200+0000", 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110717 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185251+0000", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0930000000000002e-05 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192476+0000", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072249539999999996 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192591+0000", 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115382 2024-09-21T22:50:21.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3127 3.1 e31/24 3:86733ef3:::rbd_data.111ea6ec99f1.0000000000000004:head v 31'24, mlcod=30'21)", 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6464691140000003, 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071006139999999999, 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185553+0000", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185555+0000", 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.592e-06 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185568+0000", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2787e-05 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185589+0000", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1373e-05 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185605+0000", 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5381999999999999e-05 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185742+0000", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013768400000000001 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.185806+0000", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3942999999999994e-05 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192598+0000", 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067914309999999997 2024-09-21T22:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.192654+0000", 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6422000000000003e-05 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3140 3.9 3:94fc2d77:::rbd_data.111ea6ec99f1.0000000000000018:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6380760759999999, 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090689740000000005, 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3140 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193946+0000", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193948+0000", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4929999999999999e-06 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193954+0000", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193975+0000", 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1256000000000001e-05 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193990+0000", 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4714e-05 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.197704+0000", 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003714326 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.197947+0000", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242298 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.198405+0000", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045871900000000001 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199885+0000", 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001479478 2024-09-21T22:50:21.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.202762+0000", 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028774680000000002 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.202890+0000", 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012720599999999999 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203015+0000", 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012587899999999999 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3147 3.1b e31/24 3:dea0d918:::rbd_data.111ea6ec99f1.0000000000000028:head v 31'29, mlcod=30'26)", 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6323149370000003, 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070594120000000002, 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199708+0000", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199710+0000", 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.967e-06 2024-09-21T22:50:21.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199724+0000", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.394e-05 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199755+0000", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1962999999999999e-05 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199773+0000", 2024-09-21T22:50:21.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7955999999999999e-05 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199936+0000", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000162602 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.199996+0000", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0130000000000002e-05 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.206664+0000", 2024-09-21T22:50:21.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066681650000000002 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.206767+0000", 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010268900000000001 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3151 3.13 e31/24 3:cce47d61:::rbd_data.111ea6ec99f1.0000000000000069:head v 31'41, mlcod=30'37)", 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6285165729999997, 2024-09-21T22:50:21.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072897400000000003, 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203506+0000", 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203508+0000", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9039999999999999e-06 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203524+0000", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5806000000000001e-05 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203559+0000", 2024-09-21T22:50:21.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5206000000000001e-05 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203581+0000", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2350000000000001e-05 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203701+0000", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119287 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.203756+0000", 2024-09-21T22:50:21.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5745000000000002e-05 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.210549+0000", 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067928470000000003 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.210796+0000", 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024659500000000001 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:3155 3.1b e31/24 3:dcf1d737:::rbd_data.111ea6ec99f1.000000000000011b:head v 31'31, mlcod=30'26)", 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.627807625, 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087106030000000008, 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204215+0000", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204217+0000", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204231+0000", 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4613000000000001e-05 2024-09-21T22:50:21.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204259+0000", 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7682999999999999e-05 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204279+0000", 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9599999999999999e-05 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204760+0000", 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048154300000000002 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.204816+0000", 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.63e-05 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.212749+0000", 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079325850000000007 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.212926+0000", 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017649899999999999 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3171 3.1a 3:5f8bd33a:::rbd_data.111ea6ec99f1.0000000000000210:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6186587589999997, 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087212890000000001, 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3171 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213364+0000", 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213366+0000", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.827e-06 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213374+0000", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213404+0000", 2024-09-21T22:50:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9799999999999999e-05 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.213418+0000", 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3944999999999999e-05 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.214844+0000", 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014259940000000001 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.215033+0000", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189571 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.215421+0000", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038801200000000002 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.217787+0000", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002365818 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221880+0000", 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040925290000000001 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221996+0000", 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115919 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.222085+0000", 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9351999999999994e-05 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3183 3.14 3:29e0d94b:::rbd_data.111ea6ec99f1.00000000000002b0:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6117405170000003, 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071140329999999996, 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3183 2024-09-21T22:50:21.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220282+0000", 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.220285+0000", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7259999999999998e-06 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221056+0000", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077125900000000005 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221094+0000", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7991000000000003e-05 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221116+0000", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2422999999999999e-05 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221251+0000", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134592 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221438+0000", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000187023 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:21.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.221828+0000", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038974099999999998 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.226593+0000", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047647169999999999 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227227+0000", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063445899999999998 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227316+0000", 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8633000000000003e-05 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.227396+0000", 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0469000000000005e-05 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "pg_update_log_missing(3.11 epoch 32/24 rep_tid 370 entries 32'52 (0'0) error 3:8c61b6c4:::rbd_data.111e7572a6e.000000000000006f:head by client.4382.0:4211 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'49)", 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:18.311708+0000", 2024-09-21T22:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5203144000000002, 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084653579999999992, 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311708+0000", 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311708+0000", 2024-09-21T22:50:21.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311709+0000", 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5499999999999997e-07 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311718+0000", 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3379999999999994e-06 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311741+0000", 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.313e-05 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311762+0000", 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0985e-05 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.311863+0000", 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100788 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.320174+0000", 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.008310362 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "pg_update_log_missing(3.1b epoch 32/24 rep_tid 277 entries 32'42 (0'0) error 3:db423d12:::rbd_data.111e7572a6e.0000000000000070:head by client.4382.0:4212 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'38)", 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:18.312640+0000", 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5193824199999999, 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075445520000000004, 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312640+0000", 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312640+0000", 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312641+0000", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4199999999999995e-07 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312649+0000", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6329999999999996e-06 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312665+0000", 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6265000000000001e-05 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312675+0000", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0243e-05 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312760+0000", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4722000000000006e-05 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.320185+0000", 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074249470000000003 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:4214 3.13 e32/24 3:ce0b0c97:::rbd_data.111e7572a6e.0000000000000072:head v 32'53, mlcod=32'49)", 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:18.312840+0000", 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5191828780000001, 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073826610000000004, 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312840+0000", 2024-09-21T22:50:21.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312840+0000", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312841+0000", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2399999999999997e-07 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312847+0000", 2024-09-21T22:50:21.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6669999999999999e-06 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312860+0000", 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2333e-05 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312867+0000", 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4290000000000004e-06 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312930+0000", 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3532000000000001e-05 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.312956+0000", 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5638000000000001e-05 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.320187+0000", 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072307300000000003 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.320222+0000", 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5508e-05 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:4817 3.11 e32/24 3:8a21413e:::rbd_object_map.111e20a3ef21:head v 32'162, mlcod=32'160)", 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:19.092541+0000", 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.7394810729999999, 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007706007, 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092541+0000", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092541+0000", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092543+0000", 2024-09-21T22:50:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6169999999999999e-06 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092554+0000", 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0953e-05 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092581+0000", 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6973000000000001e-05 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092594+0000", 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2957e-05 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092678+0000", 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4081999999999999e-05 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.092704+0000", 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5788000000000001e-05 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.100165+0000", 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074607809999999997 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:19.100248+0000", 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2855999999999994e-05 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:21.932 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xed37c1b 2024-09-21T22:50:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:1088 trunc from 0x5b25e7 to 0xed37c1b 2024-09-21T22:50:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:1089 punch from 0x4a6febf to 0x4a7cf49, (0xd08a bytes) 2024-09-21T22:50:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:1091 read 0x8ad0bb6 thru 0x8ad262d (0x1a78 bytes) 2024-09-21T22:50:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:1092 trunc from 0xed37c1b to 0x4c79758 2024-09-21T22:50:21.935 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:21.929+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:21.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:21.947 INFO:teuthology.orchestra.run.smithi094.stdout:1093 write 0xada2a4 thru 0xae5115 (0xae72 bytes) 2024-09-21T22:50:21.952 INFO:teuthology.orchestra.run.smithi094.stdout:1094 read 0x1dabf09 thru 0x1db856f (0xc667 bytes) 2024-09-21T22:50:21.952 INFO:teuthology.orchestra.run.smithi094.stdout:1095 punch from 0x1125e1d to 0x112c731, (0x6914 bytes) 2024-09-21T22:50:21.952 INFO:teuthology.orchestra.run.smithi094.stdout:1096 read 0x3f6695c thru 0x3f682ad (0x1952 bytes) 2024-09-21T22:50:21.978 INFO:teuthology.orchestra.run.smithi094.stdout:1097 write 0x9568dff thru 0x956e8a0 (0x5aa2 bytes) 2024-09-21T22:50:21.988 INFO:teuthology.orchestra.run.smithi094.stdout:1099 writesame 0x62849c0 thru 0x62853c4 data_size 0xf(0xa05 bytes) 2024-09-21T22:50:21.996 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:21.996 INFO:teuthology.orchestra.run.smithi094.stdout:1101 read 0x5a088ca thru 0x5a184f6 (0xfc2d bytes) 2024-09-21T22:50:22.014 INFO:teuthology.orchestra.run.smithi094.stdout:1102 write 0xb9920c0 thru 0xb99eb42 (0xca83 bytes) 2024-09-21T22:50:22.020 INFO:teuthology.orchestra.run.smithi094.stdout:1103 write 0x4f438da thru 0x4f464bb (0x2be2 bytes) 2024-09-21T22:50:22.026 INFO:teuthology.orchestra.run.smithi094.stdout:1104 punch from 0x9f19b7c to 0x9f20768, (0x6bec bytes) 2024-09-21T22:50:22.026 INFO:teuthology.orchestra.run.smithi094.stdout:1105 read 0x76007b7 thru 0x760ddf4 (0xd63e bytes) 2024-09-21T22:50:22.027 INFO:teuthology.orchestra.run.smithi094.stdout:1106 read 0xb31f8af thru 0xb32da77 (0xe1c9 bytes) 2024-09-21T22:50:22.027 INFO:teuthology.orchestra.run.smithi094.stdout:1108 write 0x558b2f8 thru 0x558e1db (0x2ee4 bytes) 2024-09-21T22:50:22.032 INFO:teuthology.orchestra.run.smithi094.stdout:1109 read 0x603876b thru 0x6041bdf (0x9475 bytes) 2024-09-21T22:50:22.033 INFO:teuthology.orchestra.run.smithi094.stdout:1110 punch from 0xaa9241 to 0xab04d6, (0x7295 bytes) 2024-09-21T22:50:22.033 INFO:teuthology.orchestra.run.smithi094.stdout:1111 read 0x29d6d0d thru 0x29e18da (0xabce bytes) 2024-09-21T22:50:22.035 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:22.029+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:22.036 INFO:teuthology.orchestra.run.smithi094.stdout:1112 trunc from 0xb99eb43 to 0xc7830a4 2024-09-21T22:50:22.048 INFO:teuthology.orchestra.run.smithi094.stdout:1113 read 0xbf8c90 thru 0xc017d2 (0x8b43 bytes) 2024-09-21T22:50:22.075 INFO:teuthology.orchestra.run.smithi094.stdout:1114 trunc from 0xc7830a4 to 0x2cf8284 2024-09-21T22:50:22.105 INFO:teuthology.orchestra.run.smithi094.stdout:1115 read 0x19ab298 thru 0x19b35f4 (0x835d bytes) 2024-09-21T22:50:22.115 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:22.115 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:22.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:22.135 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:22.129+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:22.141 INFO:teuthology.orchestra.run.smithi094.stdout:1119 write 0xb16da40 thru 0xb179d0a (0xc2cb bytes) 2024-09-21T22:50:22.146 INFO:teuthology.orchestra.run.smithi094.stdout:1121 punch from 0x44c9781 to 0x44cbba8, (0x2427 bytes) 2024-09-21T22:50:22.147 INFO:teuthology.orchestra.run.smithi094.stdout:1122 write 0xcf358e thru 0xcfec21 (0xb694 bytes) 2024-09-21T22:50:22.165 INFO:teuthology.orchestra.run.smithi094.stdout:1123 trunc from 0xb179d0b to 0x525246c 2024-09-21T22:50:22.184 INFO:teuthology.orchestra.run.smithi094.stdout:1126 read 0xfe955c thru 0xfed93b (0x43e0 bytes) 2024-09-21T22:50:22.217 INFO:teuthology.orchestra.run.smithi094.stdout:1127 write 0xce7bc84 thru 0xce87e81 (0xc1fe bytes) 2024-09-21T22:50:22.226 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:22.226 INFO:teuthology.orchestra.run.smithi094.stdout:1132 read 0x5e0bcc1 thru 0x5e170f0 (0xb430 bytes) 2024-09-21T22:50:22.226 INFO:teuthology.orchestra.run.smithi094.stdout:1135 read 0x438511f thru 0x438f28c (0xa16e bytes) 2024-09-21T22:50:22.236 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:22.234+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:22.254 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.254 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:22.254 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:22.254 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:22.254 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:22.257 INFO:teuthology.orchestra.run.smithi094.stdout:1136 trunc from 0xce87e82 to 0x153aec 2024-09-21T22:50:22.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:22.299 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:22.319 INFO:teuthology.orchestra.run.smithi094.stdout:1138 writesame 0x29f7d65 thru 0x2a01ce1 data_size 0x1(0x9f7d bytes) 2024-09-21T22:50:22.337 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:22.333+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:22.364 INFO:teuthology.orchestra.run.smithi094.stdout:1141 write 0xdf4c38e thru 0xdf57d81 (0xb9f4 bytes) 2024-09-21T22:50:22.386 INFO:teuthology.orchestra.run.smithi094.stdout:1142 trunc from 0xdf57d82 to 0x7602c20 2024-09-21T22:50:22.409 INFO:teuthology.orchestra.run.smithi094.stdout:1143 write 0x4003d41 thru 0x4006f23 (0x31e3 bytes) 2024-09-21T22:50:22.417 INFO:teuthology.orchestra.run.smithi094.stdout:1145 write 0x1ccb17e thru 0x1ccc3dc (0x125f bytes) 2024-09-21T22:50:22.424 INFO:teuthology.orchestra.run.smithi094.stdout:1146 read 0x313479e thru 0x3140dbb (0xc61e bytes) 2024-09-21T22:50:22.424 INFO:teuthology.orchestra.run.smithi094.stdout:1147 read 0x2d60ddc thru 0x2d6ad2c (0x9f51 bytes) 2024-09-21T22:50:22.424 INFO:teuthology.orchestra.run.smithi094.stdout:1149 read 0x49bb6f2 thru 0x49c3f17 (0x8826 bytes) 2024-09-21T22:50:22.424 INFO:teuthology.orchestra.run.smithi094.stdout:1150 write 0x3adf1f6 thru 0x3ae937d (0xa188 bytes) 2024-09-21T22:50:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:1151 trunc from 0x7602c20 to 0x93e98d4 2024-09-21T22:50:22.438 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:22.433+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 25.470345430999998, 2024-09-21T22:50:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012130994000000001, 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4269", 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/4252523962", 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971589+0000", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971598+0000", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1870000000000003e-06 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971609+0000", 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0783e-05 2024-09-21T22:50:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971644+0000", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.472e-05 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971670+0000", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6007e-05 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971856+0000", 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018626800000000001 2024-09-21T22:50:22.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.971893+0000", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807000000000002e-05 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.975231+0000", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00333811 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.975253+0000", 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2160000000000001e-05 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979419+0000", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041659879999999998 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979758+0000", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033847700000000001 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.979998+0000", 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000240688 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.981947+0000", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001949002 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983505+0000", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015578409999999999 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983632+0000", 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126823 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:56.983720+0000", 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8133000000000005e-05 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1188 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8108480020000002, 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085004539999999993, 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1188 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631086+0000", 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631088+0000", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1030000000000001e-06 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631094+0000", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5570000000000001e-06 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631107+0000", 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2816000000000001e-05 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631115+0000", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.632917+0000", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001802056 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.633061+0000", 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000143759 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.633680+0000", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061894499999999998 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639319+0000", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056388999999999996 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639452+0000", 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000132772 2024-09-21T22:50:22.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639526+0000", 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3773e-05 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639587+0000", 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1356e-05 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1189 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8107998589999994, 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085832630000000007, 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1189 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631135+0000", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7599999999999996e-07 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631141+0000", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631184+0000", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3043000000000001e-05 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631211+0000", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7149999999999999e-05 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.634622+0000", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034107759999999999 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.634884+0000", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026270800000000001 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.635491+0000", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00060649200000000003 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639429+0000", 2024-09-21T22:50:22.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039378099999999999 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639601+0000", 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000172566 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639665+0000", 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4215999999999996e-05 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639718+0000", 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2448999999999999e-05 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1190 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8106772939999995, 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018342312, 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1190 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631257+0000", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631258+0000", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.13e-06 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631271+0000", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2277e-05 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631296+0000", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5466e-05 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631306+0000", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6399999999999992e-06 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.636550+0000", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005243826 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.636842+0000", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029216099999999998 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.637874+0000", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001031922 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644481+0000", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066069800000000001 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649278+0000", 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047974660000000002 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649438+0000", 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159779 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649599+0000", 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016166499999999999 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1191 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8105436279999996, 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018524929999999998, 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1191 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631391+0000", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631392+0000", 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.327e-06 2024-09-21T22:50:22.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631403+0000", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1286e-05 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631417+0000", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3906999999999999e-05 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631426+0000", 2024-09-21T22:50:22.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7730000000000005e-06 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.639898+0000", 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084718320000000003 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.640159+0000", 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026053799999999999 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.641256+0000", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010972779999999999 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649043+0000", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007787459 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649654+0000", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061120499999999997 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649795+0000", 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140073 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649916+0000", 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121252 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1192 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8104915019999996, 2024-09-21T22:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018702169000000001, 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1192 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631443+0000", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631444+0000", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8700000000000005e-07 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631448+0000", 2024-09-21T22:50:22.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4070000000000003e-06 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631485+0000", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631496+0000", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1337e-05 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.642355+0000", 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010859133 2024-09-21T22:50:22.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.642573+0000", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021829799999999999 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.643635+0000", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010612989999999999 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649233+0000", 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055979560000000003 2024-09-21T22:50:22.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.649944+0000", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00071163800000000001 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650081+0000", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013650199999999999 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650145+0000", 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4362999999999993e-05 2024-09-21T22:50:22.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1193 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8104113959999992, 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018941726999999998, 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1193 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631523+0000", 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631524+0000", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1299999999999999e-07 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631534+0000", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0156999999999999e-05 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631546+0000", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.164e-05 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631554+0000", 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0199999999999994e-06 2024-09-21T22:50:22.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644532+0000", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012978686999999999 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.644662+0000", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012948899999999999 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.645392+0000", 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072992399999999996 2024-09-21T22:50:22.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650162+0000", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047703590000000001 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650304+0000", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014169599999999999 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650393+0000", 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8806000000000003e-05 2024-09-21T22:50:22.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650465+0000", 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2136000000000002e-05 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1194 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8103587129999994, 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01936705, 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1194 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631576+0000", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3399999999999998e-07 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631581+0000", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4959999999999998e-06 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631606+0000", 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5089999999999999e-05 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631614+0000", 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2300000000000008e-06 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.646403+0000", 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014788875999999999 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.646673+0000", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026966700000000002 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.647843+0000", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011701509999999999 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650628+0000", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027844990000000002 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650668+0000", 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0973000000000002e-05 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650830+0000", 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000161769 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.650943+0000", 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112565 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1195 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8102955830000003, 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027238346, 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1195 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631639+0000", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631640+0000", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4e-07 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631650+0000", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0496000000000001e-05 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631672+0000", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1250000000000002e-05 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631680+0000", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5969999999999992e-06 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.651088+0000", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019407569999999999 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.652841+0000", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017528229999999999 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.653876+0000", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010356950000000001 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658659+0000", 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047829939999999996 2024-09-21T22:50:22.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658682+0000", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2853e-05 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658761+0000", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8988999999999999e-05 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658877+0000", 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011613899999999999 2024-09-21T22:50:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1196 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8102422570000005, 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028067307, 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1196 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631692+0000", 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631693+0000", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3300000000000001e-07 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631697+0000", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3599999999999998e-06 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631709+0000", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2064e-05 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631717+0000", 2024-09-21T22:50:22.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.593e-06 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.654952+0000", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023235267 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.655198+0000", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024591100000000002 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.655888+0000", 2024-09-21T22:50:22.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068951500000000003 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.658895+0000", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030075100000000001 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659588+0000", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00069257900000000002 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659694+0000", 2024-09-21T22:50:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106099 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659760+0000", 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5675999999999999e-05 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1197 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8102034440000008, 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028784578000000002, 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1197 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631731+0000", 2024-09-21T22:50:22.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631732+0000", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631736+0000", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3429999999999998e-06 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631759+0000", 2024-09-21T22:50:22.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2877000000000001e-05 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.631767+0000", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1729999999999995e-06 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.656584+0000", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024816955000000002 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.656729+0000", 2024-09-21T22:50:22.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014541200000000001 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.657707+0000", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097799600000000007 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.659812+0000", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00210487 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660354+0000", 2024-09-21T22:50:22.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054205599999999996 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660446+0000", 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1935999999999995e-05 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:13.660516+0000", 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9282999999999994e-05 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1803 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3880512990000007, 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0097822800000000008, 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1803 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053883+0000", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053884+0000", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053888+0000", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9369999999999997e-06 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053897+0000", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3400000000000004e-06 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.053904+0000", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0700000000000001e-06 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058380+0000", 2024-09-21T22:50:22.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044756400000000003 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058513+0000", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013261499999999999 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.058837+0000", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032426900000000002 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063168+0000", 2024-09-21T22:50:22.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043308779999999998 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063526+0000", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035849599999999998 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063605+0000", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8891000000000001e-05 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.063665+0000", 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0310999999999999e-05 2024-09-21T22:50:22.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1880 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.2402467529999992, 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083682879999999998, 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1880 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201688+0000", 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201689+0000", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6400000000000001e-07 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201694+0000", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9100000000000002e-06 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201709+0000", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4348999999999999e-05 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201721+0000", 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.223e-05 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205069+0000", 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033475139999999998 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205117+0000", 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8128999999999999e-05 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205315+0000", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019795300000000001 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.208298+0000", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029836590000000001 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.209953+0000", 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016543969999999999 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210013+0000", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0393999999999998e-05 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210056+0000", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2889000000000002e-05 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:1881 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.2402004170000005, 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084419609999999996, 2024-09-21T22:50:22.515 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1881 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201734+0000", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201735+0000", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5199999999999996e-07 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201740+0000", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8779999999999997e-06 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201753+0000", 2024-09-21T22:50:22.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3026e-05 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.201762+0000", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6420000000000002e-06 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205657+0000", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038946409999999999 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205747+0000", 2024-09-21T22:50:22.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9629000000000005e-05 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.205984+0000", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000237505 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.209162+0000", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031778269999999998 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210096+0000", 2024-09-21T22:50:22.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093459700000000001 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210142+0000", 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5720000000000003e-05 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.210176+0000", 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3844000000000003e-05 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2014 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.0444170449999994, 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010643727, 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2014 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397517+0000", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397519+0000", 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0720000000000001e-06 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397524+0000", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1420000000000001e-06 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397528+0000", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7609999999999998e-06 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397532+0000", 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0600000000000001e-06 2024-09-21T22:50:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.399910+0000", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023775060000000002 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.399961+0000", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1323000000000003e-05 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400152+0000", 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000190774 2024-09-21T22:50:22.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407832+0000", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007679745 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408024+0000", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019234699999999999 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408098+0000", 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3428000000000002e-05 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408161+0000", 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3569000000000001e-05 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2015 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.0443974699999998, 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010750964999999999, 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2015 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397537+0000", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8599999999999999e-07 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397539+0000", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.782e-06 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397543+0000", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4529999999999998e-06 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.397545+0000", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6740000000000001e-06 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400728+0000", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031827919999999998 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.400780+0000", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1768000000000003e-05 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.401085+0000", 2024-09-21T22:50:22.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030565699999999998 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.407841+0000", 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067553020000000004 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408206+0000", 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000365115 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408243+0000", 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6789000000000003e-05 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.408288+0000", 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5346999999999998e-05 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2209 3.16 3:6bb22ccb:::rbd_data.111ea6ec99f1.000000000000010b:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.757534637, 2024-09-21T22:50:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082901020000000006, 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2209 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684400+0000", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684401+0000", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0500000000000003e-07 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684405+0000", 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3919999999999996e-06 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684415+0000", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6569999999999992e-06 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684420+0000", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5400000000000003e-06 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684488+0000", 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7451000000000002e-05 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.684683+0000", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019530700000000001 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.686035+0000", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013524850000000001 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692558+0000", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0065231040000000001 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692583+0000", 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4189000000000002e-05 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692628+0000", 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5809000000000001e-05 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.692690+0000", 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1463000000000004e-05 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2223 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7385506519999998, 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092793560000000008, 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2223 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703384+0000", 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.27e-07 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703387+0000", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4549999999999998e-06 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703391+0000", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5770000000000004e-06 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.703396+0000", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7929999999999997e-06 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707621+0000", 2024-09-21T22:50:22.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042251579999999997 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707719+0000", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8195000000000004e-05 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.708030+0000", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031057699999999998 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712552+0000", 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045221200000000001 2024-09-21T22:50:22.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712573+0000", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1135999999999999e-05 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712613+0000", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9835000000000001e-05 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.712663+0000", 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0182999999999997e-05 2024-09-21T22:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:2225 3.1c 3:3f3e9105:::rbd_data.111ea6ec99f1.0000000000000295:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.735101094, 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010523404, 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2225 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706833+0000", 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706835+0000", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3430000000000001e-06 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706842+0000", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1080000000000004e-06 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706858+0000", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6350000000000001e-05 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706866+0000", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.706925+0000", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9172999999999997e-05 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.707070+0000", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000144948 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.709041+0000", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019702029999999998 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717189+0000", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081485010000000007 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717240+0000", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1069999999999997e-05 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717308+0000", 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8011999999999999e-05 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:14.717357+0000", 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619999999999999e-05 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:3129 3.1d 3:bf2f0da1:::rbd_data.111ea6ec99f1.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.258273215, 2024-09-21T22:50:22.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098082800000000008, 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3129 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183661+0000", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183662+0000", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183671+0000", 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9959999999999999e-06 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183692+0000", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0662999999999999e-05 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.183708+0000", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5444e-05 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.186410+0000", 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027025320000000001 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.186651+0000", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024083199999999999 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.187201+0000", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055044200000000005 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.189161+0000", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019600149999999998 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193310+0000", 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041485610000000003 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193402+0000", 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2379000000000004e-05 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:17.193470+0000", 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.553 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1152 read 0x13e4dc7 thru 0x13eaf13 (0x614d bytes) 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1153 write 0x2914ff6 thru 0x291a0e1 (0x50ec bytes) 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1154 writesame 0xc77f659 thru 0xc780403 data_size 0x1(0xdab bytes) 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1155 write 0xe4af25 thru 0xe597de (0xe8ba bytes) 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1157 trunc from 0xc780404 to 0xe28a21 2024-09-21T22:50:22.554 INFO:teuthology.orchestra.run.smithi094.stdout:1158 write 0x49d1596 thru 0x49d5002 (0x3a6d bytes) 2024-09-21T22:50:22.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:22.555 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:22.549+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:22.581 INFO:teuthology.orchestra.run.smithi094.stdout:1159 writesame 0xa608736 thru 0xa614c05 data_size 0x2f(0xc4d0 bytes) 2024-09-21T22:50:22.597 INFO:teuthology.orchestra.run.smithi094.stdout:1160 writesame 0xb0ac46f thru 0xb0bb452 data_size 0x1a(0xefe4 bytes) 2024-09-21T22:50:22.597 INFO:teuthology.orchestra.run.smithi094.stdout:1161 write 0x209970 thru 0x20a3c3 (0xa54 bytes) 2024-09-21T22:50:22.602 INFO:teuthology.orchestra.run.smithi094.stdout:1163 write 0x1980cea thru 0x1988b71 (0x7e88 bytes) 2024-09-21T22:50:22.608 INFO:teuthology.orchestra.run.smithi094.stdout:1164 write 0xa7ef6ec thru 0xa7f5a58 (0x636d bytes) 2024-09-21T22:50:22.614 INFO:teuthology.orchestra.run.smithi094.stdout:1165 write 0x29ccbb0 thru 0x29d38f8 (0x6d49 bytes) 2024-09-21T22:50:22.618 INFO:teuthology.orchestra.run.smithi094.stdout:1168 write 0x2fe991f thru 0x2ff7bc4 (0xe2a6 bytes) 2024-09-21T22:50:22.633 INFO:teuthology.orchestra.run.smithi094.stdout:1169 trunc from 0xb0bb453 to 0xea8c590 2024-09-21T22:50:22.647 INFO:teuthology.orchestra.run.smithi094.stdout:1171 write 0x1403d64 thru 0x140d254 (0x94f1 bytes) 2024-09-21T22:50:22.651 INFO:teuthology.orchestra.run.smithi094.stdout:1173 write 0x870da3b thru 0x870f427 (0x19ed bytes) 2024-09-21T22:50:22.654 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:22.649+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:22.673 INFO:teuthology.orchestra.run.smithi094.stdout:1175 trunc from 0xea8c590 to 0x9a5716c 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:6429 3.11 3.944520f1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:22.691608+0000", 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.7033000000000001e-05, 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010511299999999999, 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6429 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691608+0000", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691608+0000", 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691610+0000", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9999999999999999e-06 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691620+0000", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0234000000000001e-05 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691647+0000", 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6279999999999999e-05 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:22.691660+0000", 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3103e-05 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-21T22:50:22.694 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:22.701 INFO:teuthology.orchestra.run.smithi094.stdout:1176 read 0x4d2821 thru 0x4d75f0 (0x4dd0 bytes) 2024-09-21T22:50:22.704 INFO:teuthology.orchestra.run.smithi094.stdout:1179 trunc from 0x9a5716c to 0x90c4c41 2024-09-21T22:50:22.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:22.749 INFO:teuthology.orchestra.run.smithi094.stdout:1181 write 0xe630af7 thru 0xe63fd71 (0xf27b bytes) 2024-09-21T22:50:22.753 INFO:teuthology.orchestra.run.smithi094.stdout:1182 write 0x6b03a24 thru 0x6b0f942 (0xbf1f bytes) 2024-09-21T22:50:22.755 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:22.749+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:22.759 INFO:teuthology.orchestra.run.smithi094.stdout:1183 punch from 0xbeeb033 to 0xbef4d3d, (0x9d0a bytes) 2024-09-21T22:50:22.760 INFO:teuthology.orchestra.run.smithi094.stdout:1185 writesame 0xa684a39 thru 0xa694895 data_size 0x1(0xfe5d bytes) 2024-09-21T22:50:22.828 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:22.828 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:22.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:22.843 INFO:teuthology.orchestra.run.smithi094.stdout:1186 write 0x24d46d0 thru 0x24d57ae (0x10df bytes) 2024-09-21T22:50:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:1188 write 0xa15b0c9 thru 0xa160e59 (0x5d91 bytes) 2024-09-21T22:50:22.855 INFO:teuthology.orchestra.run.smithi094.stdout:1189 read 0x9068a60 thru 0x9072f4f (0xa4f0 bytes) 2024-09-21T22:50:22.856 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:22.849+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:22.885 INFO:teuthology.orchestra.run.smithi094.stdout:1190 trunc from 0xe63fd72 to 0x21081b1 2024-09-21T22:50:22.908 INFO:teuthology.orchestra.run.smithi094.stdout:1191 read 0x15845f thru 0x1661ea (0xdd8c bytes) 2024-09-21T22:50:22.935 INFO:teuthology.orchestra.run.smithi094.stdout:1192 writesame 0x7a8e4cf thru 0x7a99bfb data_size 0x93(0xb72d bytes) 2024-09-21T22:50:22.957 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:22.954+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:22.960 INFO:teuthology.orchestra.run.smithi094.stdout:1193 writesame 0xc469068 thru 0xc46bbdc data_size 0x19(0x2b75 bytes) 2024-09-21T22:50:22.970 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:22.970 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.03192001, 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:22.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:22.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:22.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.912171173000001, 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013356734, 2024-09-21T22:50:22.977 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055046+0000", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5000000000000001e-07 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055053+0000", 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7630000000000003e-06 2024-09-21T22:50:22.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055065+0000", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.223e-05 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055072+0000", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4240000000000004e-06 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061686+0000", 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066135489999999998 2024-09-21T22:50:22.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061927+0000", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000241215 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.062297+0000", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036980999999999998 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.066240+0000", 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039434400000000003 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068176+0000", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001935778 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068315+0000", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000138746 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068402+0000", 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6928999999999994e-05 2024-09-21T22:50:22.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.912126043000001, 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013602839, 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:22.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055091+0000", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055184+0000", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2909000000000001e-05 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055198+0000", 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-21T22:50:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055206+0000", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3329999999999997e-06 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.063913+0000", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087066629999999999 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064150+0000", 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023710199999999999 2024-09-21T22:50:22.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064512+0000", 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036199300000000002 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068135+0000", 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036232759999999999 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068426+0000", 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000290878 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068566+0000", 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000139815 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068693+0000", 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012713000000000001 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.911991663999999, 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013916583, 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 88 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055226+0000", 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2999999999999999e-07 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055232+0000", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3029999999999996e-06 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055242+0000", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0176999999999999e-05 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055249+0000", 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067011+0000", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011761939000000001 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067152+0000", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140853 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067372+0000", 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021964099999999999 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068842+0000", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001470482 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068986+0000", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014380299999999999 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069073+0000", 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7498999999999997e-05 2024-09-21T22:50:22.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069141+0000", 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7832999999999999e-05 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.741697091999999, 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581728, 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:22.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 109 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225521+0000", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6360000000000001e-06 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225745+0000", 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022433199999999999 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225777+0000", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1955999999999997e-05 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225795+0000", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8244e-05 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233255+0000", 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074596480000000001 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233518+0000", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026267499999999997 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233943+0000", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042513700000000002 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.237933+0000", 2024-09-21T22:50:22.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039902160000000004 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241946+0000", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004013282 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242026+0000", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9926999999999996e-05 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242101+0000", 2024-09-21T22:50:22.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4674999999999999e-05 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.741370197999998, 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019620268, 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225847+0000", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3200000000000001e-06 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225858+0000", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0852999999999999e-05 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225876+0000", 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7450000000000001e-05 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225885+0000", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6900000000000004e-06 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236393+0000", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010507722000000001 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236628+0000", 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023490800000000001 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236996+0000", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036759500000000002 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241762+0000", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047658960000000004 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245313+0000", 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035513490000000001 2024-09-21T22:50:23.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245395+0000", 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2372000000000003e-05 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245466+0000", 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1112999999999995e-05 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.741313799, 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020023948999999999, 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7000000000000004e-07 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225910+0000", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6900000000000003e-06 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225931+0000", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1364000000000001e-05 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225945+0000", 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.402e-05 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238193+0000", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012247655 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238448+0000", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025524100000000002 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238841+0000", 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000392321 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241921+0000", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030808160000000001 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245516+0000", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035948149999999999 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245673+0000", 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015628500000000001 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245926+0000", 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025397199999999999 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.741245789000001, 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020395262000000001, 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225972+0000", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.19e-06 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226193+0000", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022107499999999999 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226215+0000", 2024-09-21T22:50:23.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1742999999999998e-05 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226224+0000", 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4739999999999994e-06 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239713+0000", 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013488909 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239935+0000", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022189099999999999 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.240321+0000", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038573399999999999 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245291+0000", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049707479999999997 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246237+0000", 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094542599999999997 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246307+0000", 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0568999999999995e-05 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246366+0000", 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8502999999999999e-05 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.740971798, 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020321847000000001, 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226246+0000", 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:23.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226254+0000", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226265+0000", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0293e-05 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226272+0000", 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0169999999999999e-06 2024-09-21T22:50:23.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242228+0000", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01595618 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242480+0000", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025191500000000002 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242848+0000", 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036843299999999999 2024-09-21T22:50:23.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246075+0000", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032263650000000001 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246385+0000", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031026200000000001 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246506+0000", 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121455 2024-09-21T22:50:23.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246566+0000", 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0056999999999998e-05 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.740932154999999, 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020480281999999999, 2024-09-21T22:50:23.022 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226285+0000", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226291+0000", 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.66e-06 2024-09-21T22:50:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226302+0000", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.181e-05 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226310+0000", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.267e-06 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243720+0000", 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017410454999999998 2024-09-21T22:50:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243957+0000", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023674199999999999 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.244409+0000", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045170900000000001 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246213+0000", 2024-09-21T22:50:23.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00180459 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246588+0000", 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037527699999999998 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246707+0000", 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118399 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246764+0000", 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7623e-05 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.740870292, 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024358679000000001, 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226347+0000", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8400000000000003e-07 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226468+0000", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121322 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226483+0000", 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4569999999999999e-05 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226491+0000", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0770000000000008e-06 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246868+0000", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020376894999999999 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246999+0000", 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013165200000000001 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.247197+0000", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197309 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250432+0000", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032354889999999998 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250549+0000", 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011647899999999999 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250638+0000", 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9341999999999999e-05 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250705+0000", 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6660000000000002e-05 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.74070871, 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024433862000000001, 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226509+0000", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9400000000000004e-07 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226515+0000", 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3389999999999998e-06 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226525+0000", 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9070000000000002e-06 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226532+0000", 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.866e-06 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248383+0000", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021851272000000001 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248545+0000", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016223600000000001 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248767+0000", 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222165 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250529+0000", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001761372 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250736+0000", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000207293 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250864+0000", 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000127992 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250941+0000", 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7526000000000003e-05 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.740673229999999, 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024955143999999999, 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226544+0000", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226550+0000", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226560+0000", 2024-09-21T22:50:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3270000000000007e-06 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226566+0000", 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249453+0000", 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022886468 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249587+0000", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133702 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249882+0000", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029489099999999998 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250965+0000", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010834309999999999 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251227+0000", 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000262522 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251386+0000", 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158405 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251498+0000", 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011236899999999999 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.243768042999999, 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424575, 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-21T22:50:23.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723449+0000", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8799999999999998e-07 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723452+0000", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.864e-06 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723457+0000", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723461+0000", 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4989999999999999e-06 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733711+0000", 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010250377999999999 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733854+0000", 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014302600000000001 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.734075+0000", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000221282 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735173+0000", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010972250000000001 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737475+0000", 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023023990000000001 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737557+0000", 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2229000000000002e-05 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737694+0000", 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136816 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.243750953999999, 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01520267, 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:23.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723466+0000", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3999999999999998e-07 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723468+0000", 2024-09-21T22:50:23.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4310000000000001e-06 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723472+0000", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7299999999999999e-06 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723475+0000", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9679999999999998e-06 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735404+0000", 2024-09-21T22:50:23.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011928970000000001 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735473+0000", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9604000000000006e-05 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735582+0000", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108411 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738444+0000", 2024-09-21T22:50:23.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028617740000000001 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738467+0000", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3923000000000001e-05 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738556+0000", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8955999999999993e-05 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738668+0000", 2024-09-21T22:50:23.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111663 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.243737281000001, 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015433831, 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723480+0000", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5400000000000002e-07 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723516+0000", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6826999999999998e-05 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723529+0000", 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2544000000000001e-05 2024-09-21T22:50:23.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723544+0000", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4688999999999999e-05 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735951+0000", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012407110000000001 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736089+0000", 2024-09-21T22:50:23.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137916 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736299+0000", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209929 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738732+0000", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024338570000000002 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738762+0000", 2024-09-21T22:50:23.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9442999999999999e-05 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738839+0000", 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7052999999999996e-05 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738913+0000", 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4009000000000002e-05 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.243663622, 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015612934, 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8599999999999998e-07 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723557+0000", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.177e-06 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723562+0000", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9030000000000003e-06 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723566+0000", 2024-09-21T22:50:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737789+0000", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014223372999999999 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737904+0000", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011495200000000001 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738063+0000", 2024-09-21T22:50:23.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158331 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738952+0000", 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00088979599999999999 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738987+0000", 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4440000000000002e-05 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739077+0000", 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0549000000000003e-05 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739166+0000", 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8200000000000003e-05 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:23.067 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.573076853, 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013813499999999999, 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 158 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394140+0000", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1699999999999997e-07 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394254+0000", 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113719 2024-09-21T22:50:23.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394274+0000", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0086e-05 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394288+0000", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3917000000000001e-05 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401006+0000", 2024-09-21T22:50:23.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067183939999999999 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401140+0000", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013358900000000001 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401345+0000", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020530799999999999 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403313+0000", 2024-09-21T22:50:23.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019673379999999999 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407738+0000", 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00442531 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407837+0000", 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9029000000000003e-05 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407953+0000", 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116093 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.572901881, 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013836113000000001, 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394316+0000", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.266e-06 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394326+0000", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0227000000000001e-05 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394341+0000", 2024-09-21T22:50:23.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5373e-05 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394353+0000", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2044e-05 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403365+0000", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090115799999999999 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403506+0000", 2024-09-21T22:50:23.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140712 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403703+0000", 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197431 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.405003+0000", 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00129959 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407975+0000", 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029723610000000002 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408045+0000", 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0253000000000007e-05 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408151+0000", 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105276 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:4701 3.11 3:8dd64291:::rbd_data.111e20a3ef21.00000000000000ef:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0869990349999998, 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031764344, 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4701 2024-09-21T22:50:23.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880218+0000", 2024-09-21T22:50:23.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8299999999999995e-07 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880223+0000", 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880234+0000", 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1189000000000001e-05 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880240+0000", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6080000000000003e-06 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880285+0000", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4907999999999998e-05 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880327+0000", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2394000000000001e-05 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:23.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.884093+0000", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037659909999999998 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911880+0000", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027787056000000001 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911909+0000", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9221e-05 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:23.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911946+0000", 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6439000000000001e-05 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911982+0000", 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5936000000000003e-05 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:23.084 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:23.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:23.087 INFO:teuthology.orchestra.run.smithi094.stdout:1195 trunc from 0xc46bbdd to 0x3b9428f 2024-09-21T22:50:23.087 INFO:teuthology.orchestra.run.smithi094.stdout:1196 write 0xc4ba107 thru 0xc4bf70f (0x5609 bytes) 2024-09-21T22:50:23.087 INFO:teuthology.orchestra.run.smithi094.stdout:1200 read 0x770fcc6 thru 0x77124da (0x2815 bytes) 2024-09-21T22:50:23.087 INFO:teuthology.orchestra.run.smithi094.stdout:1201 writesame 0xe4f6bde thru 0xe4fe979 data_size 0x2(0x7d9c bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1206 read 0xaf71d38 thru 0xaf7df91 (0xc25a bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1207 write 0x40a4434 thru 0x40a61eb (0x1db8 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1209 read 0xcfc26b1 thru 0xcfc726e (0x4bbe bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1210 write 0x9260fa4 thru 0x9268884 (0x78e1 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1213 write 0x2f4f0b4 thru 0x2f53fc6 (0x4f13 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1214 read 0x1f4f58 thru 0x1fe62e (0x96d7 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1215 writesame 0xb9e2901 thru 0xb9e8fd8 data_size 0xc(0x66d8 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1216 write 0x1c1a649 thru 0x1c1f57e (0x4f36 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1217 read 0xb946c8f thru 0xb951f57 (0xb2c9 bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1218 writesame 0x902d8a6 thru 0x903287f data_size 0x3(0x4fda bytes) 2024-09-21T22:50:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:1220 punch from 0x861d80c to 0x862d2d1, (0xfac5 bytes) 2024-09-21T22:50:23.088 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:23.081+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:23.089 INFO:teuthology.orchestra.run.smithi094.stdout:1221 read 0xe1aafd0 thru 0xe1b2f24 (0x7f55 bytes) 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:1222 read 0xa33d758 thru 0xa3431d4 (0x5a7d bytes) 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:1223 punch from 0xdd2631a to 0xdd3202a, (0xbd10 bytes) 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:1225 read 0x9eb5dc0 thru 0x9ebd640 (0x7881 bytes) 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:1227 read 0x44e850d thru 0x44edf5c (0x5a50 bytes) 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.090 INFO:teuthology.orchestra.run.smithi094.stdout:1231 read 0x8ccd3c3 thru 0x8cd8470 (0xb0ae bytes) 2024-09-21T22:50:23.105 INFO:teuthology.orchestra.run.smithi094.stdout:1232 trunc from 0xe4fe97a to 0x7deeea3 2024-09-21T22:50:23.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:50:23.139 INFO:teuthology.orchestra.run.smithi094.stdout:1233 trunc from 0x7deeea3 to 0xcbe1e1a 2024-09-21T22:50:23.147 INFO:teuthology.orchestra.run.smithi094.stdout:1234 writesame 0x8b25479 thru 0x8b34b2f data_size 0x25(0xf6b7 bytes) 2024-09-21T22:50:23.155 INFO:teuthology.orchestra.run.smithi094.stdout:1235 writesame 0x3b9bb47 thru 0x3ba4d7a data_size 0xc(0x9234 bytes) 2024-09-21T22:50:23.165 INFO:teuthology.orchestra.run.smithi094.stdout:1236 write 0x1278642 thru 0x1287094 (0xea53 bytes) 2024-09-21T22:50:23.167 INFO:teuthology.orchestra.run.smithi094.stdout:1237 write 0xb85d90e thru 0xb869665 (0xbd58 bytes) 2024-09-21T22:50:23.173 INFO:teuthology.orchestra.run.smithi094.stdout:1238 write 0x22ccc7d thru 0x22d0083 (0x3407 bytes) 2024-09-21T22:50:23.177 INFO:teuthology.orchestra.run.smithi094.stdout:1239 write 0x6af95ab thru 0x6afff3c (0x6992 bytes) 2024-09-21T22:50:23.179 INFO:teuthology.orchestra.run.smithi094.stdout:1240 write 0x26c4f4d thru 0x26cbadc (0x6b90 bytes) 2024-09-21T22:50:23.180 INFO:teuthology.orchestra.run.smithi094.stdout:1241 read 0x306d8e0 thru 0x30722e9 (0x4a0a bytes) 2024-09-21T22:50:23.181 INFO:teuthology.orchestra.run.smithi094.stdout:1242 write 0xe5635c thru 0xe5a825 (0x44ca bytes) 2024-09-21T22:50:23.184 INFO:teuthology.orchestra.run.smithi094.stdout:1243 read 0x903634 thru 0x911a65 (0xe432 bytes) 2024-09-21T22:50:23.186 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.186 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.186 INFO:teuthology.orchestra.run.smithi094.stdout:1247 write 0x57d582a thru 0x57dca5a (0x7231 bytes) 2024-09-21T22:50:23.186 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:23.181+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:23.207 INFO:teuthology.orchestra.run.smithi094.stdout:1248 trunc from 0xcbe1e1a to 0x29ec35c 2024-09-21T22:50:23.229 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:23.229 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:23.229 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:23.229 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:23.244 INFO:teuthology.orchestra.run.smithi094.stdout:1249 writesame 0x3372565 thru 0x3378f0c data_size 0x45(0x69a8 bytes) 2024-09-21T22:50:23.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:23.246 INFO:teuthology.orchestra.run.smithi094.stdout:1250 read 0x2a44b12 thru 0x2a4e78b (0x9c7a bytes) 2024-09-21T22:50:23.247 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.249 INFO:teuthology.orchestra.run.smithi094.stdout:1255 trunc from 0x3378f0d to 0x1e8c2a9 2024-09-21T22:50:23.263 INFO:teuthology.orchestra.run.smithi094.stdout:1256 read 0x178caa thru 0x1815b8 (0x890f bytes) 2024-09-21T22:50:23.273 INFO:teuthology.orchestra.run.smithi094.stdout:1257 trunc from 0x1e8c2a9 to 0x8c56563 2024-09-21T22:50:23.281 INFO:teuthology.orchestra.run.smithi094.stdout:1258 read 0x706d058 thru 0x7075023 (0x7fcc bytes) 2024-09-21T22:50:23.287 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:23.281+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:23.289 INFO:teuthology.orchestra.run.smithi094.stdout:1259 write 0x9371785 thru 0x937332d (0x1ba9 bytes) 2024-09-21T22:50:23.308 INFO:teuthology.orchestra.run.smithi094.stdout:1260 write 0xd17d763 thru 0xd1864fd (0x8d9b bytes) 2024-09-21T22:50:23.324 INFO:teuthology.orchestra.run.smithi094.stdout:1262 writesame 0xee6a6da thru 0xee6b27f data_size 0x15(0xba6 bytes) 2024-09-21T22:50:23.324 INFO:teuthology.orchestra.run.smithi094.stdout:1264 writesame 0xe621ca8 thru 0xe62faab data_size 0x1a(0xde04 bytes) 2024-09-21T22:50:23.333 INFO:teuthology.orchestra.run.smithi094.stdout:1265 writesame 0xea0abf1 thru 0xea164f8 data_size 0x3e(0xb908 bytes) 2024-09-21T22:50:23.339 INFO:teuthology.orchestra.run.smithi094.stdout:1267 punch from 0xe84cab1 to 0xe84f61b, (0x2b6a bytes) 2024-09-21T22:50:23.339 INFO:teuthology.orchestra.run.smithi094.stdout:1268 write 0x266c043 thru 0x2670b4f (0x4b0d bytes) 2024-09-21T22:50:23.340 INFO:teuthology.orchestra.run.smithi094.stdout:1270 read 0x3029d20 thru 0x302e2c2 (0x45a3 bytes) 2024-09-21T22:50:23.341 INFO:teuthology.orchestra.run.smithi094.stdout:1271 read 0x57fa1e5 thru 0x57fdef8 (0x3d14 bytes) 2024-09-21T22:50:23.361 INFO:teuthology.orchestra.run.smithi094.stdout:1272 trunc from 0xee6b280 to 0xe05f18 2024-09-21T22:50:23.387 INFO:teuthology.orchestra.run.smithi094.stdout:1273 punch from 0x789fff to 0x794a2d, (0xaa2e bytes) 2024-09-21T22:50:23.388 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:23.381+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:23.388 INFO:teuthology.orchestra.run.smithi094.stdout:1275 read 0x16713e thru 0x171da0 (0xac63 bytes) 2024-09-21T22:50:23.389 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:23.389 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:23.389 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:23.389 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:23.389 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:23.404 INFO:teuthology.orchestra.run.smithi094.stdout:1277 writesame 0x389f1ab thru 0x38aaf3b data_size 0x1(0xbd91 bytes) 2024-09-21T22:50:23.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:23.432 INFO:teuthology.orchestra.run.smithi094.stdout:1279 write 0xd771dd1 thru 0xd77ff62 (0xe192 bytes) 2024-09-21T22:50:23.439 INFO:teuthology.orchestra.run.smithi094.stdout:1282 write 0x8ef38af thru 0x8ef5456 (0x1ba8 bytes) 2024-09-21T22:50:23.446 INFO:teuthology.orchestra.run.smithi094.stdout:1284 write 0x9b364ee thru 0x9b43ae1 (0xd5f4 bytes) 2024-09-21T22:50:23.454 INFO:teuthology.orchestra.run.smithi094.stdout:1285 punch from 0xa277403 to 0xa278a34, (0x1631 bytes) 2024-09-21T22:50:23.454 INFO:teuthology.orchestra.run.smithi094.stdout:1286 read 0x7ee7178 thru 0x7ef282b (0xb6b4 bytes) 2024-09-21T22:50:23.454 INFO:teuthology.orchestra.run.smithi094.stdout:1287 punch from 0x33e2b6c to 0x33edb24, (0xafb8 bytes) 2024-09-21T22:50:23.454 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.463 INFO:teuthology.orchestra.run.smithi094.stdout:1291 write 0xda53e85 thru 0xda636be (0xf83a bytes) 2024-09-21T22:50:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:1292 read 0x7017149 thru 0x7024e83 (0xdd3b bytes) 2024-09-21T22:50:23.475 INFO:teuthology.orchestra.run.smithi094.stdout:1294 write 0x47fede3 thru 0x480aacb (0xbce9 bytes) 2024-09-21T22:50:23.481 INFO:teuthology.orchestra.run.smithi094.stdout:1295 punch from 0xd34652d to 0xd34fe4f, (0x9922 bytes) 2024-09-21T22:50:23.481 INFO:teuthology.orchestra.run.smithi094.stdout:1296 write 0x12836dd thru 0x12924c8 (0xedec bytes) 2024-09-21T22:50:23.483 INFO:teuthology.orchestra.run.smithi094.stdout:1297 writesame 0xbe74796 thru 0xbe772ba data_size 0x5(0x2b25 bytes) 2024-09-21T22:50:23.488 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:23.485+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:23.491 INFO:teuthology.orchestra.run.smithi094.stdout:1298 write 0xcb01d3b thru 0xcb0c029 (0xa2ef bytes) 2024-09-21T22:50:23.499 INFO:teuthology.orchestra.run.smithi094.stdout:1299 write 0x7d32bb3 thru 0x7d34539 (0x1987 bytes) 2024-09-21T22:50:23.503 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.503 INFO:teuthology.orchestra.run.smithi094.stdout:1301 read 0xb6b4d70 thru 0xb6bb7c7 (0x6a58 bytes) 2024-09-21T22:50:23.503 INFO:teuthology.orchestra.run.smithi094.stdout:1303 write 0x21eedf6 thru 0x21f0789 (0x1994 bytes) 2024-09-21T22:50:23.505 INFO:teuthology.orchestra.run.smithi094.stdout:1304 read 0xc09f181 thru 0xc0ade7b (0xecfb bytes) 2024-09-21T22:50:23.520 INFO:teuthology.orchestra.run.smithi094.stdout:1305 trunc from 0xda636bf to 0x4940e32 2024-09-21T22:50:23.561 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:23.561 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:23.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:23.561 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1667 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6883769330000007, 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017652939999999999, 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1667 2024-09-21T22:50:23.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868501+0000", 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868503+0000", 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4090000000000001e-06 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868510+0000", 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3799999999999996e-06 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868514+0000", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0280000000000002e-06 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886013+0000", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017498841000000001 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886073+0000", 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0015000000000001e-05 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886153+0000", 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9800000000000002e-05 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1668 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6883574059999997, 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017819267999999999, 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1668 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2599999999999998e-07 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868522+0000", 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7260000000000001e-06 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868527+0000", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8010000000000003e-06 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868530+0000", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886193+0000", 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017663239000000001 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886254+0000", 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0977000000000002e-05 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886339+0000", 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4622000000000004e-05 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.570 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1672 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6883424829999996, 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017989136999999999, 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1672 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.96e-07 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868536+0000", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.517e-06 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868541+0000", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5059999999999997e-06 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868545+0000", 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9009999999999996e-06 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886380+0000", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017834843 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886443+0000", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3443999999999998e-05 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886524+0000", 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-05 2024-09-21T22:50:23.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1673 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6883279330000001, 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018154611000000001, 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1673 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868550+0000", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868552+0000", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2589999999999999e-06 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868558+0000", 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2040000000000002e-06 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868564+0000", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4489999999999996e-06 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886564+0000", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017999549 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886625+0000", 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1026999999999997e-05 2024-09-21T22:50:23.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886704+0000", 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8805000000000002e-05 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1674 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6883053029999999, 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018316941999999999, 2024-09-21T22:50:23.579 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1674 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3000000000000002e-07 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868574+0000", 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6929999999999999e-06 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868579+0000", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1510000000000003e-06 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868583+0000", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8139999999999998e-06 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886746+0000", 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018163190999999999 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886808+0000", 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1564999999999998e-05 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886889+0000", 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1198000000000004e-05 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.583 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1675 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6882897329999995, 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018479580999999998, 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1675 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868588+0000", 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.91e-07 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868589+0000", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.576e-06 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868598+0000", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.551e-06 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868602+0000", 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.794e-06 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886929+0000", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018327209000000001 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886992+0000", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3553000000000006e-05 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887067+0000", 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4606999999999999e-05 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.587 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1676 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6882506710000005, 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018629687999999998, 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1676 2024-09-21T22:50:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868627+0000", 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-07 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868629+0000", 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1789999999999998e-06 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868636+0000", 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868642+0000", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3579999999999999e-06 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887104+0000", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018462545 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887176+0000", 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1540999999999998e-05 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887256+0000", 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0141000000000004e-05 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1694 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6881679760000008, 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018819487999999999, 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1694 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9700000000000003e-07 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868712+0000", 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1619999999999998e-06 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868716+0000", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6099999999999999e-06 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868720+0000", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887298+0000", 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018577750000000001 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887385+0000", 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7077000000000007e-05 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887529+0000", 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014399200000000001 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1695 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.596 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6881495930000003, 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01924435, 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1695 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:23.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868730+0000", 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4830000000000002e-06 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868757+0000", 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6149999999999999e-05 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868762+0000", 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9849999999999997e-06 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887608+0000", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018846471 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887781+0000", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017245300000000001 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887972+0000", 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019134100000000001 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1699 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6881079929999991, 2024-09-21T22:50:23.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019655261, 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1699 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:23.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868770+0000", 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0699999999999998e-07 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868771+0000", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9300000000000002e-06 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868777+0000", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1270000000000002e-06 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868780+0000", 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6830000000000001e-06 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888128+0000", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019347907000000001 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888265+0000", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013713700000000001 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888424+0000", 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015907000000000001 2024-09-21T22:50:23.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1700 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6880924159999999, 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019917859999999999, 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1700 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0199999999999998e-07 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868786+0000", 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4750000000000001e-06 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868793+0000", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868799+0000", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.596e-06 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888513+0000", 2024-09-21T22:50:23.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019714381999999999 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888606+0000", 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2514000000000002e-05 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888703+0000", 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6559999999999997e-05 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1701 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6880699719999992, 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020220140000000001, 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1701 2024-09-21T22:50:23.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868808+0000", 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868810+0000", 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868816+0000", 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868820+0000", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888804+0000", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019984927 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888894+0000", 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9239000000000004e-05 2024-09-21T22:50:23.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889027+0000", 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133566 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1704 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6880526269999994, 2024-09-21T22:50:23.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020436185999999999, 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1704 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868825+0000", 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868826+0000", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.454e-06 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868831+0000", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6009999999999997e-06 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868836+0000", 2024-09-21T22:50:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5230000000000003e-06 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889078+0000", 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020241278000000001 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889176+0000", 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8041000000000005e-05 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889261+0000", 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4946999999999998e-05 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1705 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6879676700000008, 2024-09-21T22:50:23.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020613012999999999, 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1705 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:23.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868910+0000", 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868914+0000", 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9310000000000001e-06 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868925+0000", 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0586e-05 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868933+0000", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9689999999999994e-06 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889307+0000", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020373901 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889399+0000", 2024-09-21T22:50:23.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2146999999999997e-05 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889522+0000", 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012362500000000001 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1706 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.625 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6879305490000007, 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021099340000000001, 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1706 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868950+0000", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9869999999999999e-06 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868959+0000", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9069999999999996e-06 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868965+0000", 2024-09-21T22:50:23.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889619+0000", 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020653120000000001 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889851+0000", 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023190799999999999 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890046+0000", 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000195389 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1707 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6879024289999993, 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022001837, 2024-09-21T22:50:23.630 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1707 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2500000000000005e-07 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868978+0000", 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8949999999999998e-06 2024-09-21T22:50:23.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868989+0000", 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1241e-05 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868996+0000", 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6499999999999999e-06 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890125+0000", 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021128457 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890278+0000", 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015349800000000001 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890977+0000", 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069847099999999999 2024-09-21T22:50:23.634 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1708 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6878709139999994, 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022287839, 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1708 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869007+0000", 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5000000000000003e-07 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869010+0000", 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9469999999999999e-06 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869020+0000", 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0316000000000001e-05 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869027+0000", 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5939999999999999e-06 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891011+0000", 2024-09-21T22:50:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021984786999999999 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891138+0000", 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012628800000000001 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891294+0000", 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015635700000000001 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1807 3.12 3:4e819a64:::rbd_data.111ea6ec99f1.000000000000009f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4921337609999998, 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027460622000000001, 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1807 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064747+0000", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7929999999999999e-06 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064763+0000", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5603999999999998e-05 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064797+0000", 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4319999999999997e-05 2024-09-21T22:50:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064833+0000", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5666e-05 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065001+0000", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016830799999999999 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065362+0000", 2024-09-21T22:50:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036063099999999998 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.070173+0000", 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048112459999999999 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092106+0000", 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021932953000000002 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092120+0000", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4318000000000001e-05 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092162+0000", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1980999999999997e-05 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092204+0000", 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1801999999999997e-05 2024-09-21T22:50:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3757 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.3363152930000002, 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017800281000000001, 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3757 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220563+0000", 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220568+0000", 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3299999999999998e-06 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220582+0000", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4047e-05 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220592+0000", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5570000000000001e-06 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230061+0000", 2024-09-21T22:50:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094694610000000002 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230174+0000", 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011302300000000001 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230809+0000", 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063446900000000003 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232849+0000", 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002040395 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238192+0000", 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053432089999999998 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238279+0000", 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6624000000000003e-05 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238362+0000", 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3295999999999997e-05 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3758 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.3362633759999998, 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017995779, 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3758 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220615+0000", 2024-09-21T22:50:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2500000000000004e-07 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220620+0000", 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5709999999999997e-06 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220636+0000", 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6030999999999999e-05 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220647+0000", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0411000000000001e-05 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231339+0000", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010691849999999999 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231406+0000", 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7846000000000001e-05 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231663+0000", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025653299999999998 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.233751+0000", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020885869999999998 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238457+0000", 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047057230000000002 2024-09-21T22:50:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238543+0000", 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6191000000000002e-05 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238610+0000", 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6111000000000005e-05 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:23.659 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:23.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:23.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:1306 writesame 0x81e514b thru 0x81e6f73 data_size 0x1(0x1e29 bytes) 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:1308 write 0x2124c5d thru 0x2131f4f (0xd2f3 bytes) 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:1311 write 0xbfd90e0 thru 0xbfe06ba (0x75db bytes) 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:1312 punch from 0x16b54de to 0x16bdf8c, (0x8aae bytes) 2024-09-21T22:50:23.662 INFO:teuthology.orchestra.run.smithi094.stdout:1313 writesame 0x1bd9dec thru 0x1bdfcc7 data_size 0x1a(0x5edc bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1314 write 0xd34090d thru 0xd347504 (0x6bf8 bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1315 writesame 0x63e30c6 thru 0x63efc16 data_size 0x1f(0xcb51 bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1317 read 0x7cd0a7d thru 0x7cd5181 (0x4705 bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1321 read 0x1768643 thru 0x1771040 (0x89fe bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1322 read 0xb9b5cf5 thru 0xb9c1ce3 (0xbfef bytes) 2024-09-21T22:50:23.663 INFO:teuthology.orchestra.run.smithi094.stdout:1323 trunc from 0xd347505 to 0x7c94ab4 2024-09-21T22:50:23.663 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:23.653+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:23.677 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.677 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.692 INFO:teuthology.orchestra.run.smithi094.stdout:1326 write 0x87a827d thru 0x87b5eaf (0xdc33 bytes) 2024-09-21T22:50:23.720 INFO:teuthology.orchestra.run.smithi094.stdout:1329 write 0xdded6a3 thru 0xddf93db (0xbd39 bytes) 2024-09-21T22:50:23.727 INFO:teuthology.orchestra.run.smithi094.stdout:1330 punch from 0x7c30976 to 0x7c3fbb3, (0xf23d bytes) 2024-09-21T22:50:23.727 INFO:teuthology.orchestra.run.smithi094.stdout:1331 punch from 0x319d7f to 0x320799, (0x6a1a bytes) 2024-09-21T22:50:23.727 INFO:teuthology.orchestra.run.smithi094.stdout:1333 read 0xb768747 thru 0xb76ef85 (0x683f bytes) 2024-09-21T22:50:23.727 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.728 INFO:teuthology.orchestra.run.smithi094.stdout:1337 writesame 0xaddcead thru 0xadec2fd data_size 0x7(0xf451 bytes) 2024-09-21T22:50:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:1338 trunc from 0xddf93dc to 0x5b7022b 2024-09-21T22:50:23.761 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:23.758+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:23.773 INFO:teuthology.orchestra.run.smithi094.stdout:1339 read 0x21c0e2e thru 0x21c3a8d (0x2c60 bytes) 2024-09-21T22:50:23.774 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:23.774 INFO:teuthology.orchestra.run.smithi094.stdout:1341 read 0xf73130 thru 0xf818c7 (0xe798 bytes) 2024-09-21T22:50:23.775 INFO:teuthology.orchestra.run.smithi094.stdout:1343 write 0x36a18c9 thru 0x36aab03 (0x923b bytes) 2024-09-21T22:50:23.779 INFO:teuthology.orchestra.run.smithi094.stdout:1345 read 0x437185f thru 0x437850d (0x6caf bytes) 2024-09-21T22:50:23.792 INFO:teuthology.orchestra.run.smithi094.stdout:1346 write 0x7142ee3 thru 0x7144f32 (0x2050 bytes) 2024-09-21T22:50:23.799 INFO:teuthology.orchestra.run.smithi094.stdout:1347 read 0x59fd212 thru 0x59ff9a4 (0x2793 bytes) 2024-09-21T22:50:23.799 INFO:teuthology.orchestra.run.smithi094.stdout:1348 read 0x36b6d8e thru 0x36bc08a (0x52fd bytes) 2024-09-21T22:50:23.800 INFO:teuthology.orchestra.run.smithi094.stdout:1350 read 0x5fa8116 thru 0x5fb4aa8 (0xc993 bytes) 2024-09-21T22:50:23.801 INFO:teuthology.orchestra.run.smithi094.stdout:1352 read 0x52b777e thru 0x52bcb30 (0x53b3 bytes) 2024-09-21T22:50:23.806 INFO:teuthology.orchestra.run.smithi094.stdout:1354 trunc from 0x7144f33 to 0x3c414f8 2024-09-21T22:50:23.827 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:23.827 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:23.827 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:23.827 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:23.832 INFO:teuthology.orchestra.run.smithi094.stdout:1356 read 0x27dafee thru 0x27e9e9d (0xeeb0 bytes) 2024-09-21T22:50:23.833 INFO:teuthology.orchestra.run.smithi094.stdout:1357 punch from 0x379e9b8 to 0x379f7ee, (0xe36 bytes) 2024-09-21T22:50:23.833 INFO:teuthology.orchestra.run.smithi094.stdout:1359 write 0x29ac9a4 thru 0x29ae4e0 (0x1b3d bytes) 2024-09-21T22:50:23.837 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:23.837 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:23.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:23.855 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:23.858 INFO:teuthology.orchestra.run.smithi094.stdout:1360 write 0xdc474e3 thru 0xdc4f61d (0x813b bytes) 2024-09-21T22:50:23.861 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:23.857+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:1361 writesame 0x221dd6 thru 0x22f7a7 data_size 0x31(0xd9d2 bytes) 2024-09-21T22:50:23.870 INFO:teuthology.orchestra.run.smithi094.stdout:1362 write 0x7229a86 thru 0x722bdbe (0x2339 bytes) 2024-09-21T22:50:23.874 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:23.874 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:23.876 INFO:teuthology.orchestra.run.smithi094.stdout:1363 punch from 0x3dc851e to 0x3dd2626, (0xa108 bytes) 2024-09-21T22:50:23.877 INFO:teuthology.orchestra.run.smithi094.stdout:1364 trunc from 0xdc4f61e to 0xed22e29 2024-09-21T22:50:23.886 INFO:teuthology.orchestra.run.smithi094.stdout:1365 write 0x7597fb6 thru 0x759d19b (0x51e6 bytes) 2024-09-21T22:50:23.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:23.893 INFO:teuthology.orchestra.run.smithi094.stdout:1367 writesame 0x53bea50 thru 0x53cd060 data_size 0x1(0xe611 bytes) 2024-09-21T22:50:23.916 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:23.963 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:23.958+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:23.972 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:23.973 INFO:teuthology.orchestra.run.smithi094.stdout:1368 read 0xe37544e thru 0xe381183 (0xbd36 bytes) 2024-09-21T22:50:23.974 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:23.974 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:23.994 INFO:teuthology.orchestra.run.smithi094.stdout:1369 trunc from 0xed22e29 to 0x1a490fc 2024-09-21T22:50:23.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:24.001 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:24.001 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:24.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:24.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:24.018 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.018 INFO:teuthology.orchestra.run.smithi094.stdout:1372 read 0xa0c7ce thru 0xa100bf (0x38f2 bytes) 2024-09-21T22:50:24.019 INFO:teuthology.orchestra.run.smithi094.stdout:1373 read 0x1702585 thru 0x170615a (0x3bd6 bytes) 2024-09-21T22:50:24.046 INFO:teuthology.orchestra.run.smithi094.stdout:1375 writesame 0xd325c96 thru 0xd32723f data_size 0x2(0x15aa bytes) 2024-09-21T22:50:24.046 INFO:teuthology.orchestra.run.smithi094.stdout:1377 write 0x9d3f21a thru 0x9d4bd93 (0xcb7a bytes) 2024-09-21T22:50:24.052 INFO:teuthology.orchestra.run.smithi094.stdout:1378 read 0xb4d7ce4 thru 0xb4e1538 (0x9855 bytes) 2024-09-21T22:50:24.063 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:24.057+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:24.064 INFO:teuthology.orchestra.run.smithi094.stdout:1379 writesame 0xec687aa thru 0xec73646 data_size 0x1(0xae9d bytes) 2024-09-21T22:50:24.064 INFO:teuthology.orchestra.run.smithi094.stdout:1381 read 0xebae362 thru 0xebb0e9f (0x2b3e bytes) 2024-09-21T22:50:24.088 INFO:teuthology.orchestra.run.smithi094.stdout:1382 trunc from 0xec73647 to 0xb8f560 2024-09-21T22:50:24.116 INFO:teuthology.orchestra.run.smithi094.stdout:1383 trunc from 0xb8f560 to 0x6ca80c 2024-09-21T22:50:24.118 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:50:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.165 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:24.162+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:24.171 INFO:teuthology.orchestra.run.smithi094.stdout:1385 write 0x95fc754 thru 0x9606c3a (0xa4e7 bytes) 2024-09-21T22:50:24.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:24.197 INFO:teuthology.orchestra.run.smithi094.stdout:1386 write 0xbfec95f thru 0xbffa243 (0xd8e5 bytes) 2024-09-21T22:50:24.218 INFO:teuthology.orchestra.run.smithi094.stdout:1389 trunc from 0xbffa244 to 0x2fbfea9 2024-09-21T22:50:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:1391 read 0x416b1a thru 0x41e1d6 (0x76bd bytes) 2024-09-21T22:50:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:1392 trunc from 0x2fbfea9 to 0xa05ce7e 2024-09-21T22:50:24.265 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:24.261+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:24.282 INFO:teuthology.orchestra.run.smithi094.stdout:1393 writesame 0xe1f5e31 thru 0xe1fe953 data_size 0x1f(0x8b23 bytes) 2024-09-21T22:50:24.282 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.282 INFO:teuthology.orchestra.run.smithi094.stdout:1396 writesame 0x154143d thru 0x15416cf data_size 0x1(0x293 bytes) 2024-09-21T22:50:24.287 INFO:teuthology.orchestra.run.smithi094.stdout:1397 punch from 0x100ed4f to 0x10198bb, (0xab6c bytes) 2024-09-21T22:50:24.294 INFO:teuthology.orchestra.run.smithi094.stdout:1398 trunc from 0xe1fe954 to 0xa9282ee 2024-09-21T22:50:24.306 INFO:teuthology.orchestra.run.smithi094.stdout:1399 trunc from 0xa9282ee to 0x8cfb1fd 2024-09-21T22:50:24.317 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.318 INFO:teuthology.orchestra.run.smithi094.stdout:1401 read 0x5763c1e thru 0x5769288 (0x566b bytes) 2024-09-21T22:50:24.325 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.325 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.339 INFO:teuthology.orchestra.run.smithi094.stdout:1402 write 0xde8be14 thru 0xde90eca (0x50b7 bytes) 2024-09-21T22:50:24.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:1403 punch from 0x9d69bbb to 0x9d6b5b9, (0x19fe bytes) 2024-09-21T22:50:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.348 INFO:teuthology.orchestra.run.smithi094.stdout:1406 read 0xb9bb8ac thru 0xb9bc2c7 (0xa1c bytes) 2024-09-21T22:50:24.348 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.348 INFO:teuthology.orchestra.run.smithi094.stdout:1409 writesame 0x1375369 thru 0x137e066 data_size 0x1(0x8cfe bytes) 2024-09-21T22:50:24.366 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:24.362+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:24.392 INFO:teuthology.orchestra.run.smithi094.stdout:1410 writesame 0x8df743b thru 0x8df7ee9 data_size 0x1(0xaaf bytes) 2024-09-21T22:50:24.399 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.402 INFO:teuthology.orchestra.run.smithi094.stdout:1412 trunc from 0xde90ecb to 0xc6cbb49 2024-09-21T22:50:24.428 INFO:teuthology.orchestra.run.smithi094.stdout:1413 write 0xed7e049 thru 0xed8af4d (0xcf05 bytes) 2024-09-21T22:50:24.435 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.448 INFO:teuthology.orchestra.run.smithi094.stdout:1416 trunc from 0xed8af4e to 0x692b6ff 2024-09-21T22:50:24.466 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:24.461+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:24.481 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.481 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.485 INFO:teuthology.orchestra.run.smithi094.stdout:1419 trunc from 0x692b6ff to 0xcda4295 2024-09-21T22:50:24.498 INFO:teuthology.orchestra.run.smithi094.stdout:1421 read 0x4c12554 thru 0x4c1d72a (0xb1d7 bytes) 2024-09-21T22:50:24.498 INFO:teuthology.orchestra.run.smithi094.stdout:1423 punch from 0x18d04ab to 0x18d14e5, (0x103a bytes) 2024-09-21T22:50:24.498 INFO:teuthology.orchestra.run.smithi094.stdout:1424 read 0x38f5d4f thru 0x390486f (0xeb21 bytes) 2024-09-21T22:50:24.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:24.499 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.499 INFO:teuthology.orchestra.run.smithi094.stdout:1426 write 0xbbfa9bf thru 0xbbfbbfd (0x123f bytes) 2024-09-21T22:50:24.508 INFO:teuthology.orchestra.run.smithi094.stdout:1427 writesame 0x8e3e0ce thru 0x8e448a6 data_size 0xd(0x67d9 bytes) 2024-09-21T22:50:24.517 INFO:teuthology.orchestra.run.smithi094.stdout:1428 read 0x6bfba2e thru 0x6c063f9 (0xa9cc bytes) 2024-09-21T22:50:24.517 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.517 INFO:teuthology.orchestra.run.smithi094.stdout:1432 write 0x13ba8de thru 0x13c8f45 (0xe668 bytes) 2024-09-21T22:50:24.520 INFO:teuthology.orchestra.run.smithi094.stdout:1433 write 0xbded0d2 thru 0xbdf52db (0x820a bytes) 2024-09-21T22:50:24.526 INFO:teuthology.orchestra.run.smithi094.stdout:1434 punch from 0x6f27a91 to 0x6f377f7, (0xfd66 bytes) 2024-09-21T22:50:24.526 INFO:teuthology.orchestra.run.smithi094.stdout:1436 read 0x92591c4 thru 0x9266356 (0xd193 bytes) 2024-09-21T22:50:24.529 INFO:teuthology.orchestra.run.smithi094.stdout:1437 trunc from 0xcda4295 to 0xb305c72 2024-09-21T22:50:24.540 INFO:teuthology.orchestra.run.smithi094.stdout:1438 read 0x3df4250 thru 0x3df68cd (0x267e bytes) 2024-09-21T22:50:24.540 INFO:teuthology.orchestra.run.smithi094.stdout:1439 writesame 0x72a3632 thru 0x72a884d data_size 0xa(0x521c bytes) 2024-09-21T22:50:24.559 INFO:teuthology.orchestra.run.smithi094.stdout:1440 write 0xd17ac40 thru 0xd17f351 (0x4712 bytes) 2024-09-21T22:50:24.561 INFO:teuthology.orchestra.run.smithi094.stdout:1441 write 0xe88021 thru 0xe8cf39 (0x4f19 bytes) 2024-09-21T22:50:24.565 INFO:teuthology.orchestra.run.smithi094.stdout:1442 write 0x2335298 thru 0x233625b (0xfc4 bytes) 2024-09-21T22:50:24.566 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:24.561+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:24.568 INFO:teuthology.orchestra.run.smithi094.stdout:1443 read 0x904bf08 thru 0x9051115 (0x520e bytes) 2024-09-21T22:50:24.569 INFO:teuthology.orchestra.run.smithi094.stdout:1444 write 0x27444b2 thru 0x274ffdd (0xbb2c bytes) 2024-09-21T22:50:24.572 INFO:teuthology.orchestra.run.smithi094.stdout:1445 read 0x581533f thru 0x58217b3 (0xc475 bytes) 2024-09-21T22:50:24.572 INFO:teuthology.orchestra.run.smithi094.stdout:1446 writesame 0x6d7ecb1 thru 0x6d868a1 data_size 0x1(0x7bf1 bytes) 2024-09-21T22:50:24.619 INFO:teuthology.orchestra.run.smithi094.stdout:1447 writesame 0x86fd32e thru 0x870a51f data_size 0x9a(0xd1f2 bytes) 2024-09-21T22:50:24.628 INFO:teuthology.orchestra.run.smithi094.stdout:1448 writesame 0x153204a thru 0x1539d82 data_size 0x1(0x7d39 bytes) 2024-09-21T22:50:24.637 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.637 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:24.661 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:24.661 INFO:teuthology.orchestra.run.smithi094.stdout:1450 write 0x6ef5839 thru 0x6f014b2 (0xbc7a bytes) 2024-09-21T22:50:24.667 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:24.662+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:24.668 INFO:teuthology.orchestra.run.smithi094.stdout:1452 punch from 0x41e7f4b to 0x41ed5d2, (0x5687 bytes) 2024-09-21T22:50:24.668 INFO:teuthology.orchestra.run.smithi094.stdout:1453 write 0x49e6a3f thru 0x49eca89 (0x604b bytes) 2024-09-21T22:50:24.673 INFO:teuthology.orchestra.run.smithi094.stdout:1455 trunc from 0xd17f352 to 0xb7efa7b 2024-09-21T22:50:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xed3c265 2024-09-21T22:50:24.690 INFO:teuthology.orchestra.run.smithi094.stdout:1459 trunc from 0xb7efa7b to 0xed3c265 2024-09-21T22:50:24.699 INFO:teuthology.orchestra.run.smithi094.stdout:1460 read 0x2274137 thru 0x227a649 (0x6513 bytes) 2024-09-21T22:50:24.700 INFO:teuthology.orchestra.run.smithi094.stdout:1462 write 0x4bce1ed thru 0x4bd6b14 (0x8928 bytes) 2024-09-21T22:50:24.703 INFO:teuthology.orchestra.run.smithi094.stdout:1464 write 0xa534b03 thru 0xa539306 (0x4804 bytes) 2024-09-21T22:50:24.711 INFO:teuthology.orchestra.run.smithi094.stdout:1465 write 0xfd6ce8 thru 0xfde13a (0x7453 bytes) 2024-09-21T22:50:24.714 INFO:teuthology.orchestra.run.smithi094.stdout:1466 writesame 0x22e5ac7 thru 0x22f36fa data_size 0x22(0xdc34 bytes) 2024-09-21T22:50:24.718 INFO:teuthology.orchestra.run.smithi094.stdout:1469 writesame 0x855eab9 thru 0x85656df data_size 0xb(0x6c27 bytes) 2024-09-21T22:50:24.727 INFO:teuthology.orchestra.run.smithi094.stdout:1470 write 0x3c2464d thru 0x3c25f19 (0x18cd bytes) 2024-09-21T22:50:24.730 INFO:teuthology.orchestra.run.smithi094.stdout:1471 write 0x7576fa5 thru 0x757e47a (0x74d6 bytes) 2024-09-21T22:50:24.763 INFO:teuthology.orchestra.run.smithi094.stdout:1472 trunc from 0xed3c265 to 0x2dbc409 2024-09-21T22:50:24.769 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:24.762+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:24.793 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.793 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.796 INFO:teuthology.orchestra.run.smithi094.stdout:1473 read 0x199d486 thru 0x19a2ac4 (0x563f bytes) 2024-09-21T22:50:24.797 INFO:teuthology.orchestra.run.smithi094.stdout:1474 punch from 0xe40180 to 0xe42f78, (0x2df8 bytes) 2024-09-21T22:50:24.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:24.809 INFO:teuthology.orchestra.run.smithi094.stdout:1476 write 0x33e8ef6 thru 0x33ebc54 (0x2d5f bytes) 2024-09-21T22:50:24.813 INFO:teuthology.orchestra.run.smithi094.stdout:1477 punch from 0x19df6c1 to 0x19e04bf, (0xdfe bytes) 2024-09-21T22:50:24.819 INFO:teuthology.orchestra.run.smithi094.stdout:1478 trunc from 0x33ebc55 to 0x4adbdb 2024-09-21T22:50:24.869 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:24.866+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:24.880 INFO:teuthology.orchestra.run.smithi094.stdout:1479 write 0x9a2ae3c thru 0x9a36921 (0xbae6 bytes) 2024-09-21T22:50:24.896 INFO:teuthology.orchestra.run.smithi094.stdout:1480 write 0x9a86a97 thru 0x9a873d7 (0x941 bytes) 2024-09-21T22:50:24.899 INFO:teuthology.orchestra.run.smithi094.stdout:1482 read 0x9a2c50c thru 0x9a33c33 (0x7728 bytes) 2024-09-21T22:50:24.910 INFO:teuthology.orchestra.run.smithi094.stdout:1483 trunc from 0x9a873d8 to 0x1626ff0 2024-09-21T22:50:24.941 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:24.941 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:24.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:24.971 INFO:teuthology.orchestra.run.smithi094.stdout:1484 writesame 0x6cbfa14 thru 0x6cca216 data_size 0x1b(0xa803 bytes) 2024-09-21T22:50:24.971 INFO:teuthology.orchestra.run.smithi094.stdout:1486 read 0x236bf56 thru 0x236f901 (0x39ac bytes) 2024-09-21T22:50:24.971 INFO:teuthology.orchestra.run.smithi094.stdout:1487 punch from 0x638eb29 to 0x6398cc6, (0xa19d bytes) 2024-09-21T22:50:24.971 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:24.965+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:24.988 INFO:teuthology.orchestra.run.smithi094.stdout:1488 write 0xbf017df thru 0xbf0f4fb (0xdd1d bytes) 2024-09-21T22:50:24.999 INFO:teuthology.orchestra.run.smithi094.stdout:1489 read 0x3d05a74 thru 0x3d130d4 (0xd661 bytes) 2024-09-21T22:50:24.999 INFO:teuthology.orchestra.run.smithi094.stdout:1491 read 0x3e6b454 thru 0x3e6bd2d (0x8da bytes) 2024-09-21T22:50:25.000 INFO:teuthology.orchestra.run.smithi094.stdout:1493 clone 6 order 21 su 65536 sc 4 2024-09-21T22:50:25.071 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:25.065+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:25.094 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.095 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:25.172 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:25.165+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:25.248 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.248 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:25.313 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:25.310+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:25.335 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone5 from 0xbf0f4fc (overlap 0x153aec) to 0x10fbf0 2024-09-21T22:50:25.374 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:25.369+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:25.393 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.393 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:25.430 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-21T22:50:25.430 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:25.430 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-21T22:50:25.430 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T22:50:25.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T22:50:25.450 INFO:tasks.ceph.osd.6:Started 2024-09-21T22:50:25.451 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:25.475 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:25.469+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:25.576 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:25.569+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:25.578 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.578 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:no valid command found; 10 closest matches: 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:0 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:1 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:abort 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:assert 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:config diff 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:config diff get 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:config get 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:config help [] 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:config set ... 2024-09-21T22:50:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:admin_socket: invalid command 2024-09-21T22:50:25.607 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T22:50:25.607 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-21T22:50:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T22:50:25.629 INFO:teuthology.orchestra.run.smithi094.stdout:1494 write 0x11812c2 thru 0x118d704 (0xc443 bytes) 2024-09-21T22:50:25.648 INFO:teuthology.orchestra.run.smithi094.stdout:1495 read 0x365fefa thru 0x3663d2b (0x3e32 bytes) 2024-09-21T22:50:25.649 INFO:teuthology.orchestra.run.smithi094.stdout:1496 read 0x8a4bf7b thru 0x8a4cb50 (0xbd6 bytes) 2024-09-21T22:50:25.649 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.649 INFO:teuthology.orchestra.run.smithi094.stdout:1499 punch from 0x84d43d2 to 0x84defa8, (0xabd6 bytes) 2024-09-21T22:50:25.649 INFO:teuthology.orchestra.run.smithi094.stdout:1502 punch from 0x8676f11 to 0x867bfe2, (0x50d1 bytes) 2024-09-21T22:50:25.668 INFO:teuthology.orchestra.run.smithi094.stdout:1504 write 0xe959da5 thru 0xe96025d (0x64b9 bytes) 2024-09-21T22:50:25.676 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:25.674+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:25.678 INFO:teuthology.orchestra.run.smithi094.stdout:1506 read 0x9ffddf0 thru 0xa00d6ba (0xf8cb bytes) 2024-09-21T22:50:25.678 INFO:teuthology.orchestra.run.smithi094.stdout:1508 write 0xd2a2f65 thru 0xd2b10cc (0xe168 bytes) 2024-09-21T22:50:25.685 INFO:teuthology.orchestra.run.smithi094.stdout:1509 read 0x2dd341d thru 0x2ddcddc (0x99c0 bytes) 2024-09-21T22:50:25.685 INFO:teuthology.orchestra.run.smithi094.stdout:1511 read 0x9090fba thru 0x909eef6 (0xdf3d bytes) 2024-09-21T22:50:25.685 INFO:teuthology.orchestra.run.smithi094.stdout:1513 punch from 0x2ec705b to 0x2ed01bc, (0x9161 bytes) 2024-09-21T22:50:25.716 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.716 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.720 INFO:teuthology.orchestra.run.smithi094.stdout:1514 trunc from 0xe96025e to 0x18b376c 2024-09-21T22:50:25.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:25.755 INFO:teuthology.orchestra.run.smithi094.stdout:1515 read 0x60134c thru 0x610c8e (0xf943 bytes) 2024-09-21T22:50:25.777 INFO:teuthology.orchestra.run.smithi094.stdout:1517 write 0x64f6ea5 thru 0x64fa288 (0x33e4 bytes) 2024-09-21T22:50:25.777 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:25.773+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:25.794 INFO:teuthology.orchestra.run.smithi094.stdout:1518 write 0x6a74da1 thru 0x6a84582 (0xf7e2 bytes) 2024-09-21T22:50:25.802 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.803 INFO:teuthology.orchestra.run.smithi094.stdout:1521 punch from 0x89eed4 to 0x8a3520, (0x464c bytes) 2024-09-21T22:50:25.811 INFO:teuthology.orchestra.run.smithi094.stdout:1522 trunc from 0x6a84583 to 0x35fcc9b 2024-09-21T22:50:25.868 INFO:teuthology.orchestra.run.smithi094.stdout:1523 write 0xc7c780c thru 0xc7d1d56 (0xa54b bytes) 2024-09-21T22:50:25.876 INFO:teuthology.orchestra.run.smithi094.stdout:1524 write 0x86c757f thru 0x86c9042 (0x1ac4 bytes) 2024-09-21T22:50:25.878 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:25.873+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:1525 punch from 0x9a09ecf to 0x9a168ac, (0xc9dd bytes) 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:1533 read 0x6d29530 thru 0x6d32d8a (0x985b bytes) 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:1537 punch from 0x16c81bd to 0x16d4270, (0xc0b3 bytes) 2024-09-21T22:50:25.882 INFO:teuthology.orchestra.run.smithi094.stdout:1539 write 0x8b6c519 thru 0x8b78eb5 (0xc99d bytes) 2024-09-21T22:50:25.890 INFO:teuthology.orchestra.run.smithi094.stdout:1540 punch from 0x9c22f2b to 0x9c30f1b, (0xdff0 bytes) 2024-09-21T22:50:25.891 INFO:teuthology.orchestra.run.smithi094.stdout:1542 punch from 0x4c33880 to 0x4c386f4, (0x4e74 bytes) 2024-09-21T22:50:25.891 INFO:teuthology.orchestra.run.smithi094.stdout:1544 writesame 0x5e0e0c7 thru 0x5e16036 data_size 0x8(0x7f70 bytes) 2024-09-21T22:50:25.899 INFO:teuthology.orchestra.run.smithi094.stdout:1545 write 0x6fbfe4e thru 0x6fc320e (0x33c1 bytes) 2024-09-21T22:50:25.900 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:25.900 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:25.908 INFO:teuthology.orchestra.run.smithi094.stdout:1549 write 0x238a0a5 thru 0x238bf6a (0x1ec6 bytes) 2024-09-21T22:50:25.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:25.917 INFO:teuthology.orchestra.run.smithi094.stdout:1552 trunc from 0xc7d1d57 to 0xba70051 2024-09-21T22:50:25.938 INFO:teuthology.orchestra.run.smithi094.stdout:1553 punch from 0xafb1cc to 0xb09d55, (0xeb89 bytes) 2024-09-21T22:50:25.938 INFO:teuthology.orchestra.run.smithi094.stdout:1554 write 0xdf5e22 thru 0xdf64e7 (0x6c6 bytes) 2024-09-21T22:50:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:25.949 INFO:teuthology.orchestra.run.smithi094.stdout:1557 trunc from 0xba70051 to 0xb27d5c1 2024-09-21T22:50:25.961 INFO:teuthology.orchestra.run.smithi094.stdout:1558 writesame 0xacc1e9f thru 0xacc8e15 data_size 0xd(0x6f77 bytes) 2024-09-21T22:50:25.974 INFO:teuthology.orchestra.run.smithi094.stdout:1559 trunc from 0xb27d5c1 to 0x8c39811 2024-09-21T22:50:25.978 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:25.973+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:25.997 INFO:teuthology.orchestra.run.smithi094.stdout:1560 read 0x10e33a0 thru 0x10e4c84 (0x18e5 bytes) 2024-09-21T22:50:25.997 INFO:teuthology.orchestra.run.smithi094.stdout:1561 read 0xe0ee7e thru 0xe1e408 (0xf58b bytes) 2024-09-21T22:50:26.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:26.010 INFO:teuthology.orchestra.run.smithi094.stdout:1562 trunc from 0x8c39811 to 0xd620846 2024-09-21T22:50:26.023 INFO:teuthology.orchestra.run.smithi094.stdout:1563 read 0x9b46f3d thru 0x9b50301 (0x93c5 bytes) 2024-09-21T22:50:26.023 INFO:teuthology.orchestra.run.smithi094.stdout:1566 write 0xae1d993 thru 0xae1dbef (0x25d bytes) 2024-09-21T22:50:26.029 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.029 INFO:teuthology.orchestra.run.smithi094.stdout:1569 read 0xd1eb73b thru 0xd1ed029 (0x18ef bytes) 2024-09-21T22:50:26.029 INFO:teuthology.orchestra.run.smithi094.stdout:1570 write 0x1aa7a61 thru 0x1aaf589 (0x7b29 bytes) 2024-09-21T22:50:26.032 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.033 INFO:teuthology.orchestra.run.smithi094.stdout:1574 trunc from 0xd620846 to 0xdb0842b 2024-09-21T22:50:26.039 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:26.039 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:26.044 INFO:teuthology.orchestra.run.smithi094.stdout:1576 read 0x17406d thru 0x180546 (0xc4da bytes) 2024-09-21T22:50:26.045 INFO:teuthology.orchestra.run.smithi094.stdout:1579 write 0xce6fc7 thru 0xcea78e (0x37c8 bytes) 2024-09-21T22:50:26.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:26.054 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.055 INFO:teuthology.orchestra.run.smithi094.stdout:1581 read 0x3874aba thru 0x387e965 (0x9eac bytes) 2024-09-21T22:50:26.055 INFO:teuthology.orchestra.run.smithi094.stdout:1582 write 0x719826d thru 0x71a2f3b (0xaccf bytes) 2024-09-21T22:50:26.060 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.060 INFO:teuthology.orchestra.run.smithi094.stdout:1584 write 0xc4e413e thru 0xc4e6382 (0x2245 bytes) 2024-09-21T22:50:26.064 INFO:teuthology.orchestra.run.smithi094.stdout:1585 write 0x11b7d10 thru 0x11c5e1c (0xe10d bytes) 2024-09-21T22:50:26.071 INFO:teuthology.orchestra.run.smithi094.stdout:1586 punch from 0x261be72 to 0x262538e, (0x951c bytes) 2024-09-21T22:50:26.071 INFO:teuthology.orchestra.run.smithi094.stdout:1587 punch from 0x1c2ce45 to 0x1c3643e, (0x95f9 bytes) 2024-09-21T22:50:26.071 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.071 INFO:teuthology.orchestra.run.smithi094.stdout:1589 punch from 0x674e64 to 0x6783a6, (0x3542 bytes) 2024-09-21T22:50:26.093 INFO:teuthology.orchestra.run.smithi094.stdout:1591 trunc from 0xdb0842b to 0x437df4a 2024-09-21T22:50:26.100 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:26.098+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:26.117 INFO:teuthology.orchestra.run.smithi094.stdout:1592 punch from 0x17e7f21 to 0x17f1f38, (0xa017 bytes) 2024-09-21T22:50:26.117 INFO:teuthology.orchestra.run.smithi094.stdout:1593 read 0x18d1969 thru 0x18d6369 (0x4a01 bytes) 2024-09-21T22:50:26.118 INFO:teuthology.orchestra.run.smithi094.stdout:1595 punch from 0x2037dd6 to 0x2043ba5, (0xbdcf bytes) 2024-09-21T22:50:26.120 INFO:teuthology.orchestra.run.smithi094.stdout:1597 trunc from 0x437df4a to 0x5270dd8 2024-09-21T22:50:26.130 INFO:teuthology.orchestra.run.smithi094.stdout:1599 read 0x2042a33 thru 0x204661c (0x3bea bytes) 2024-09-21T22:50:26.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:50:26.151 INFO:teuthology.orchestra.run.smithi094.stdout:1600 writesame 0xd76a9be thru 0xd777e3a data_size 0x13(0xd47d bytes) 2024-09-21T22:50:26.152 INFO:teuthology.orchestra.run.smithi094.stdout:1601 write 0x2c301ad thru 0x2c36562 (0x63b6 bytes) 2024-09-21T22:50:26.156 INFO:teuthology.orchestra.run.smithi094.stdout:1603 write 0x929689e thru 0x9297db8 (0x151b bytes) 2024-09-21T22:50:26.161 INFO:teuthology.orchestra.run.smithi094.stdout:1604 read 0x9e6eaf8 thru 0x9e7ce16 (0xe31f bytes) 2024-09-21T22:50:26.161 INFO:teuthology.orchestra.run.smithi094.stdout:1605 punch from 0xbb2998e to 0xbb3184a, (0x7ebc bytes) 2024-09-21T22:50:26.161 INFO:teuthology.orchestra.run.smithi094.stdout:1606 read 0x89e8486 thru 0x89f1070 (0x8beb bytes) 2024-09-21T22:50:26.161 INFO:teuthology.orchestra.run.smithi094.stdout:1607 writesame 0x464baae thru 0x46556bb data_size 0x5e(0x9c0e bytes) 2024-09-21T22:50:26.168 INFO:teuthology.orchestra.run.smithi094.stdout:1608 write 0x24c59ad thru 0x24caf06 (0x555a bytes) 2024-09-21T22:50:26.170 INFO:teuthology.orchestra.run.smithi094.stdout:1609 read 0x5cadfee thru 0x5cb18d5 (0x38e8 bytes) 2024-09-21T22:50:26.170 INFO:teuthology.orchestra.run.smithi094.stdout:1611 write 0x5bfb94a thru 0x5bfc499 (0xb50 bytes) 2024-09-21T22:50:26.173 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:26.173 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:26.174 INFO:teuthology.orchestra.run.smithi094.stdout:1612 punch from 0x3415538 to 0x3416b91, (0x1659 bytes) 2024-09-21T22:50:26.174 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.175 INFO:teuthology.orchestra.run.smithi094.stdout:1615 write 0x91e06e3 thru 0x91eef2c (0xe84a bytes) 2024-09-21T22:50:26.179 INFO:teuthology.orchestra.run.smithi094.stdout:1616 write 0x4793ad2 thru 0x47a0f34 (0xd463 bytes) 2024-09-21T22:50:26.180 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:26.174+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:26.183 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:26.183 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:26.183 INFO:teuthology.orchestra.run.smithi094.stdout:1617 writesame 0x79e14b1 thru 0x79eeba6 data_size 0x5(0xd6f6 bytes) 2024-09-21T22:50:26.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:26.191 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:26.195 INFO:teuthology.orchestra.run.smithi094.stdout:1621 writesame 0xc0ffea2 thru 0xc10a3d9 data_size 0x44(0xa538 bytes) 2024-09-21T22:50:26.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:26.210 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:26.219 INFO:teuthology.orchestra.run.smithi094.stdout:1622 trunc from 0xd777e3b to 0x47d10c 2024-09-21T22:50:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:26.231 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:26.255 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:26.268 INFO:teuthology.orchestra.run.smithi094.stdout:1624 writesame 0x974a6ca thru 0x97563c0 data_size 0x9(0xbcf7 bytes) 2024-09-21T22:50:26.277 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:26.281 INFO:teuthology.orchestra.run.smithi094.stdout:1625 write 0xdc656cb thru 0xdc72790 (0xd0c6 bytes) 2024-09-21T22:50:26.284 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:26.282+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:26.286 INFO:teuthology.orchestra.run.smithi094.stdout:1628 write 0xc6dddd8 thru 0xc6e169c (0x38c5 bytes) 2024-09-21T22:50:26.291 INFO:teuthology.orchestra.run.smithi094.stdout:1629 punch from 0x1792e41 to 0x1797482, (0x4641 bytes) 2024-09-21T22:50:26.291 INFO:teuthology.orchestra.run.smithi094.stdout:1630 write 0x65c5f2 thru 0x65ebae (0x25bd bytes) 2024-09-21T22:50:26.304 INFO:teuthology.orchestra.run.smithi094.stdout:1631 trunc from 0xdc72791 to 0x635d97a 2024-09-21T22:50:26.305 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:26.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:26.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:26.326 INFO:teuthology.orchestra.run.smithi094.stdout:1632 trunc from 0x635d97a to 0xbd54a83 2024-09-21T22:50:26.333 INFO:teuthology.orchestra.run.smithi094.stdout:1633 write 0x246d9f1 thru 0x24791e3 (0xb7f3 bytes) 2024-09-21T22:50:26.338 INFO:teuthology.orchestra.run.smithi094.stdout:1634 read 0x8332f63 thru 0x833f6a0 (0xc73e bytes) 2024-09-21T22:50:26.338 INFO:teuthology.orchestra.run.smithi094.stdout:1635 write 0x67ce4a8 thru 0x67d4df8 (0x6951 bytes) 2024-09-21T22:50:26.343 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.344 INFO:teuthology.orchestra.run.smithi094.stdout:1640 write 0x5bb0134 thru 0x5bba831 (0xa6fe bytes) 2024-09-21T22:50:26.348 INFO:teuthology.orchestra.run.smithi094.stdout:1641 writesame 0x8675f13 thru 0x86801c6 data_size 0x9c(0xa2b4 bytes) 2024-09-21T22:50:26.366 INFO:teuthology.orchestra.run.smithi094.stdout:1642 write 0xdbda137 thru 0xdbded37 (0x4c01 bytes) 2024-09-21T22:50:26.369 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:26.369 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:26.369 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:26.369 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:26.369 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:26.371 INFO:teuthology.orchestra.run.smithi094.stdout:1645 punch from 0x2361a45 to 0x23649e6, (0x2fa1 bytes) 2024-09-21T22:50:26.371 INFO:teuthology.orchestra.run.smithi094.stdout:1646 read 0x975c06f thru 0x976bbbf (0xfb51 bytes) 2024-09-21T22:50:26.371 INFO:teuthology.orchestra.run.smithi094.stdout:1647 write 0xa0e60b1 thru 0xa0ec0f8 (0x6048 bytes) 2024-09-21T22:50:26.381 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:26.377+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:26.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:26.391 INFO:teuthology.orchestra.run.smithi094.stdout:1649 trunc from 0xdbded38 to 0x4775d2c 2024-09-21T22:50:26.438 INFO:teuthology.orchestra.run.smithi094.stdout:1650 write 0xe3518fb thru 0xe351e50 (0x556 bytes) 2024-09-21T22:50:26.457 INFO:teuthology.orchestra.run.smithi094.stdout:1651 trunc from 0xe351e51 to 0x65e3650 2024-09-21T22:50:26.474 INFO:teuthology.orchestra.run.smithi094.stdout:1652 read 0x13c77e thru 0x145fc8 (0x984b bytes) 2024-09-21T22:50:26.475 INFO:teuthology.orchestra.run.smithi094.stdout:1655 write 0x29b2968 thru 0x29b9e36 (0x74cf bytes) 2024-09-21T22:50:26.480 INFO:teuthology.orchestra.run.smithi094.stdout:1656 trunc from 0x65e3650 to 0x61834fc 2024-09-21T22:50:26.481 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:26.477+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:26.488 INFO:teuthology.orchestra.run.smithi094.stdout:1658 trunc from 0x61834fc to 0x69822f6 2024-09-21T22:50:26.512 INFO:teuthology.orchestra.run.smithi094.stdout:1659 writesame 0xc083a08 thru 0xc08b091 data_size 0x2(0x768a bytes) 2024-09-21T22:50:26.512 INFO:teuthology.orchestra.run.smithi094.stdout:1660 read 0x75c2b6a thru 0x75cd186 (0xa61d bytes) 2024-09-21T22:50:26.512 INFO:teuthology.orchestra.run.smithi094.stdout:1662 write 0xb873405 thru 0xb87c595 (0x9191 bytes) 2024-09-21T22:50:26.518 INFO:teuthology.orchestra.run.smithi094.stdout:1663 punch from 0x4672bf0 to 0x46763ad, (0x37bd bytes) 2024-09-21T22:50:26.518 INFO:teuthology.orchestra.run.smithi094.stdout:1664 read 0xb46b260 thru 0xb46cd2c (0x1acd bytes) 2024-09-21T22:50:26.518 INFO:teuthology.orchestra.run.smithi094.stdout:1666 writesame 0x2c65d18 thru 0x2c6722e data_size 0x1(0x1517 bytes) 2024-09-21T22:50:26.526 INFO:teuthology.orchestra.run.smithi094.stdout:1667 write 0xa861e39 thru 0xa870cf0 (0xeeb8 bytes) 2024-09-21T22:50:26.552 INFO:teuthology.orchestra.run.smithi094.stdout:1668 trunc from 0xc08b092 to 0x28569f0 2024-09-21T22:50:26.557 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.620329243999997, 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.500580407000001, 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013356734, 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86 2024-09-21T22:50:26.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055045+0000", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055046+0000", 2024-09-21T22:50:26.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5000000000000001e-07 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055053+0000", 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7630000000000003e-06 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055065+0000", 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.223e-05 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055072+0000", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4240000000000004e-06 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061686+0000", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066135489999999998 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.061927+0000", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000241215 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.062297+0000", 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036980999999999998 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.066240+0000", 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039434400000000003 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068176+0000", 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001935778 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068315+0000", 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000138746 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068402+0000", 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6928999999999994e-05 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.500535276999997, 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013602839, 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055090+0000", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055091+0000", 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055184+0000", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2909000000000001e-05 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055198+0000", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055206+0000", 2024-09-21T22:50:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3329999999999997e-06 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.063913+0000", 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087066629999999999 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064150+0000", 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023710199999999999 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.064512+0000", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036199300000000002 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068135+0000", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036232759999999999 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068426+0000", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000290878 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068566+0000", 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000139815 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068693+0000", 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012713000000000001 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:26.576 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.500400898000002, 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013916583, 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 88 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055225+0000", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055226+0000", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2999999999999999e-07 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055232+0000", 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3029999999999996e-06 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055242+0000", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0176999999999999e-05 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.055249+0000", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067011+0000", 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011761939000000001 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067152+0000", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140853 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.067372+0000", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021964099999999999 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068842+0000", 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001470482 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.068986+0000", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014380299999999999 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069073+0000", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7498999999999997e-05 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.069141+0000", 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7832999999999999e-05 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.330106325999999, 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581728, 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 109 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225519+0000", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225521+0000", 2024-09-21T22:50:26.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6360000000000001e-06 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225745+0000", 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022433199999999999 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225777+0000", 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1955999999999997e-05 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225795+0000", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8244e-05 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233255+0000", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074596480000000001 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233518+0000", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026267499999999997 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.233943+0000", 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042513700000000002 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.237933+0000", 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039902160000000004 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241946+0000", 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004013282 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242026+0000", 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9926999999999996e-05 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242101+0000", 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4674999999999999e-05 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329779432000002, 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019620268, 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225846+0000", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225847+0000", 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3200000000000001e-06 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225858+0000", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0852999999999999e-05 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225876+0000", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7450000000000001e-05 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225885+0000", 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6900000000000004e-06 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236393+0000", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010507722000000001 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236628+0000", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023490800000000001 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.236996+0000", 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036759500000000002 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241762+0000", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047658960000000004 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245313+0000", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035513490000000001 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245395+0000", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2372000000000003e-05 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245466+0000", 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1112999999999995e-05 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329723033, 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020023948999999999, 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225903+0000", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7000000000000004e-07 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225910+0000", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6900000000000003e-06 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225931+0000", 2024-09-21T22:50:26.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1364000000000001e-05 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225945+0000", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.402e-05 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238193+0000", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012247655 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238448+0000", 2024-09-21T22:50:26.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025524100000000002 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.238841+0000", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000392321 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.241921+0000", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030808160000000001 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245516+0000", 2024-09-21T22:50:26.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035948149999999999 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245673+0000", 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015628500000000001 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245926+0000", 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025397199999999999 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329655023000001, 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020395262000000001, 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225971+0000", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.225972+0000", 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.19e-06 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226193+0000", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022107499999999999 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226215+0000", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1742999999999998e-05 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226224+0000", 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4739999999999994e-06 2024-09-21T22:50:26.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239713+0000", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013488909 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.239935+0000", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022189099999999999 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.240321+0000", 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038573399999999999 2024-09-21T22:50:26.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.245291+0000", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049707479999999997 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246237+0000", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094542599999999997 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246307+0000", 2024-09-21T22:50:26.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0568999999999995e-05 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246366+0000", 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8502999999999999e-05 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329381032000001, 2024-09-21T22:50:26.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020321847000000001, 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226245+0000", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226246+0000", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226254+0000", 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:50:26.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226265+0000", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0293e-05 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226272+0000", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0169999999999999e-06 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242228+0000", 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01595618 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242480+0000", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025191500000000002 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.242848+0000", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036843299999999999 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246075+0000", 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032263650000000001 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246385+0000", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031026200000000001 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246506+0000", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121455 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246566+0000", 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0056999999999998e-05 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329341389, 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020480281999999999, 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.610 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226284+0000", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226285+0000", 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-07 2024-09-21T22:50:26.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226291+0000", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.66e-06 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226302+0000", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.181e-05 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226310+0000", 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.267e-06 2024-09-21T22:50:26.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243720+0000", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017410454999999998 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.243957+0000", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023674199999999999 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.244409+0000", 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045170900000000001 2024-09-21T22:50:26.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246213+0000", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00180459 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246588+0000", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037527699999999998 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246707+0000", 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118399 2024-09-21T22:50:26.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246764+0000", 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7623e-05 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329279526000001, 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024358679000000001, 2024-09-21T22:50:26.615 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226346+0000", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226347+0000", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8400000000000003e-07 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226468+0000", 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121322 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226483+0000", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4569999999999999e-05 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226491+0000", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0770000000000008e-06 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246868+0000", 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020376894999999999 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.246999+0000", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013165200000000001 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.247197+0000", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197309 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250432+0000", 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032354889999999998 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250549+0000", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011647899999999999 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250638+0000", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9341999999999999e-05 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250705+0000", 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6660000000000002e-05 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329117943999996, 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024433862000000001, 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.621 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226508+0000", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226509+0000", 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9400000000000004e-07 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226515+0000", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3389999999999998e-06 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226525+0000", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9070000000000002e-06 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226532+0000", 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.866e-06 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248383+0000", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021851272000000001 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248545+0000", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016223600000000001 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.248767+0000", 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222165 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250529+0000", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001761372 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250736+0000", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000207293 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250864+0000", 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000127992 2024-09-21T22:50:26.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250941+0000", 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7526000000000003e-05 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.329082464000003, 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024955143999999999, 2024-09-21T22:50:26.626 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226543+0000", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226544+0000", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226550+0000", 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226560+0000", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3270000000000007e-06 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.226566+0000", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249453+0000", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022886468 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249587+0000", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133702 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.249882+0000", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029489099999999998 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.250965+0000", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010834309999999999 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251227+0000", 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000262522 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251386+0000", 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158405 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.251498+0000", 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011236899999999999 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.832177277, 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424575, 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.632 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723448+0000", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723449+0000", 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8799999999999998e-07 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723452+0000", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.864e-06 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723457+0000", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723461+0000", 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4989999999999999e-06 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733711+0000", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010250377999999999 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.733854+0000", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014302600000000001 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.734075+0000", 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000221282 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735173+0000", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010972250000000001 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737475+0000", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023023990000000001 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737557+0000", 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2229000000000002e-05 2024-09-21T22:50:26.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737694+0000", 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136816 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.832160188, 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01520267, 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723465+0000", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723466+0000", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3999999999999998e-07 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723468+0000", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4310000000000001e-06 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723472+0000", 2024-09-21T22:50:26.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7299999999999999e-06 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723475+0000", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9679999999999998e-06 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735404+0000", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011928970000000001 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735473+0000", 2024-09-21T22:50:26.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9604000000000006e-05 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735582+0000", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108411 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738444+0000", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028617740000000001 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738467+0000", 2024-09-21T22:50:26.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3923000000000001e-05 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738556+0000", 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8955999999999993e-05 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738668+0000", 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111663 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.832146515000002, 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015433831, 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723479+0000", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723480+0000", 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5400000000000002e-07 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723516+0000", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6826999999999998e-05 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723529+0000", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2544000000000001e-05 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723544+0000", 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4688999999999999e-05 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.735951+0000", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012407110000000001 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736089+0000", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137916 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.736299+0000", 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209929 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738732+0000", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024338570000000002 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738762+0000", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9442999999999999e-05 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738839+0000", 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7052999999999996e-05 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738913+0000", 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4009000000000002e-05 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.832072856, 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015612934, 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:26.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723553+0000", 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8599999999999998e-07 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723557+0000", 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.177e-06 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723562+0000", 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9030000000000003e-06 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.723566+0000", 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737789+0000", 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014223372999999999 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.737904+0000", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011495200000000001 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738063+0000", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000158331 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738952+0000", 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00088979599999999999 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.738987+0000", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4440000000000002e-05 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739077+0000", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0549000000000003e-05 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:54.739166+0000", 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8200000000000003e-05 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.161486087, 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013813499999999999, 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 158 2024-09-21T22:50:26.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394139+0000", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394140+0000", 2024-09-21T22:50:26.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1699999999999997e-07 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394254+0000", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113719 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394274+0000", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0086e-05 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394288+0000", 2024-09-21T22:50:26.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3917000000000001e-05 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401006+0000", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067183939999999999 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401140+0000", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013358900000000001 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.401345+0000", 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020530799999999999 2024-09-21T22:50:26.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403313+0000", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019673379999999999 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407738+0000", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00442531 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407837+0000", 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9029000000000003e-05 2024-09-21T22:50:26.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407953+0000", 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116093 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.161311115, 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013836113000000001, 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394314+0000", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394316+0000", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.266e-06 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394326+0000", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0227000000000001e-05 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394341+0000", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5373e-05 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.394353+0000", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2044e-05 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403365+0000", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090115799999999999 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403506+0000", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140712 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.403703+0000", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000197431 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.405003+0000", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00129959 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.407975+0000", 2024-09-21T22:50:26.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029723610000000002 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408045+0000", 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0253000000000007e-05 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:55.408151+0000", 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105276 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:4701 3.11 3:8dd64291:::rbd_data.111e20a3ef21.00000000000000ef:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.6754082690000001, 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031764344, 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4701 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880217+0000", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880218+0000", 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8299999999999995e-07 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880223+0000", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880234+0000", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1189000000000001e-05 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880240+0000", 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6080000000000003e-06 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880285+0000", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4907999999999998e-05 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.880327+0000", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2394000000000001e-05 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.884093+0000", 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037659909999999998 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911880+0000", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027787056000000001 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911909+0000", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9221e-05 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911946+0000", 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6439000000000001e-05 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:18.911982+0000", 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5936000000000003e-05 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:26.672 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:26.666+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:26.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:26.672 INFO:teuthology.orchestra.run.smithi094.stdout:1670 punch from 0x1d457b3 to 0x1d53ea8, (0xe6f5 bytes) 2024-09-21T22:50:26.672 INFO:teuthology.orchestra.run.smithi094.stdout:1671 trunc from 0x28569f0 to 0x719e218 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1675 punch from 0x3346c68 to 0x3352baf, (0xbf47 bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1676 write 0x93ef281 thru 0x93ef4ab (0x22b bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1677 read 0x8563ebc thru 0x856733f (0x3484 bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1679 write 0xd06b4c0 thru 0xd073b33 (0x8674 bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1681 write 0x36cf92f thru 0x36dea99 (0xf16b bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1682 writesame 0xd8e3676 thru 0xd8ec826 data_size 0x13(0x91b1 bytes) 2024-09-21T22:50:26.673 INFO:teuthology.orchestra.run.smithi094.stdout:1683 write 0xb39f875 thru 0xb3a9a8b (0xa217 bytes) 2024-09-21T22:50:26.714 INFO:teuthology.orchestra.run.smithi094.stdout:1684 trunc from 0xd8ec827 to 0x11503ed 2024-09-21T22:50:26.736 INFO:teuthology.orchestra.run.smithi094.stdout:1685 punch from 0xe4340b to 0xe47dc5, (0x49ba bytes) 2024-09-21T22:50:26.736 INFO:teuthology.orchestra.run.smithi094.stdout:1686 writesame 0x114b08 thru 0x1185f8 data_size 0x1(0x3af1 bytes) 2024-09-21T22:50:26.764 INFO:teuthology.orchestra.run.smithi094.stdout:1687 trunc from 0x11503ed to 0x50f2a0a 2024-09-21T22:50:26.772 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:26.769+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:26.776 INFO:teuthology.orchestra.run.smithi094.stdout:1689 read 0x1fcfa4a thru 0x1fd4401 (0x49b8 bytes) 2024-09-21T22:50:26.776 INFO:teuthology.orchestra.run.smithi094.stdout:1690 write 0x1cdbf07 thru 0x1cdd822 (0x191c bytes) 2024-09-21T22:50:26.782 INFO:teuthology.orchestra.run.smithi094.stdout:1695 read 0x416bbd2 thru 0x417afdb (0xf40a bytes) 2024-09-21T22:50:26.782 INFO:teuthology.orchestra.run.smithi094.stdout:1696 punch from 0x18d00fd to 0x18d371b, (0x361e bytes) 2024-09-21T22:50:26.810 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:26.810 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:26.810 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:26.810 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:26.812 INFO:teuthology.orchestra.run.smithi094.stdout:1697 writesame 0xd239a35 thru 0xd2448c6 data_size 0x29(0xae92 bytes) 2024-09-21T22:50:26.812 INFO:teuthology.orchestra.run.smithi094.stdout:1700 read 0xb9913ba thru 0xb9924d8 (0x111f bytes) 2024-09-21T22:50:26.812 INFO:teuthology.orchestra.run.smithi094.stdout:1701 write 0x90e18aa thru 0x90ee984 (0xd0db bytes) 2024-09-21T22:50:26.819 INFO:teuthology.orchestra.run.smithi094.stdout:1702 read 0x42bb6ef thru 0x42c388b (0x819d bytes) 2024-09-21T22:50:26.822 INFO:teuthology.orchestra.run.smithi094.stdout:1703 trunc from 0xd2448c7 to 0xbec7ac9 2024-09-21T22:50:26.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:26.841 INFO:teuthology.orchestra.run.smithi094.stdout:1705 write 0xc1669e1 thru 0xc171af2 (0xb112 bytes) 2024-09-21T22:50:26.849 INFO:teuthology.orchestra.run.smithi094.stdout:1706 write 0x8bfa2f7 thru 0x8c01668 (0x7372 bytes) 2024-09-21T22:50:26.856 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:26.859 INFO:teuthology.orchestra.run.smithi094.stdout:1708 trunc from 0xc171af3 to 0xdd2d930 2024-09-21T22:50:26.868 INFO:teuthology.orchestra.run.smithi094.stdout:1709 clone 7 order 21 su 32768 sc 10 2024-09-21T22:50:26.873 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:26.870+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:26.957 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:26.957 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:26.957 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:26.957 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:26.957 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:26.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:26.974 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:26.970+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:27.075 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:27.070+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:27.147 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:27.147 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:27.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:27.147 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1667 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272127467000001, 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017652939999999999, 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1667 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868501+0000", 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:27.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868503+0000", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4090000000000001e-06 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868510+0000", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3799999999999996e-06 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868514+0000", 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0280000000000002e-06 2024-09-21T22:50:27.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886013+0000", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017498841000000001 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886073+0000", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0015000000000001e-05 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886153+0000", 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9800000000000002e-05 2024-09-21T22:50:27.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1668 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.27210794, 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017819267999999999, 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1668 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2599999999999998e-07 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868522+0000", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7260000000000001e-06 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868527+0000", 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8010000000000003e-06 2024-09-21T22:50:27.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868530+0000", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886193+0000", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017663239000000001 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886254+0000", 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0977000000000002e-05 2024-09-21T22:50:27.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886339+0000", 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4622000000000004e-05 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1672 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272093017, 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017989136999999999, 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1672 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.96e-07 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868536+0000", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.517e-06 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868541+0000", 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5059999999999997e-06 2024-09-21T22:50:27.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868545+0000", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9009999999999996e-06 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886380+0000", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017834843 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886443+0000", 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3443999999999998e-05 2024-09-21T22:50:27.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886524+0000", 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-05 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1673 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272078467, 2024-09-21T22:50:27.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018154611000000001, 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1673 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868550+0000", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868552+0000", 2024-09-21T22:50:27.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2589999999999999e-06 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868558+0000", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2040000000000002e-06 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868564+0000", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4489999999999996e-06 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886564+0000", 2024-09-21T22:50:27.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017999549 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886625+0000", 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1026999999999997e-05 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886704+0000", 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8805000000000002e-05 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1674 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272055837, 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018316941999999999, 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1674 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3000000000000002e-07 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868574+0000", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6929999999999999e-06 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868579+0000", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1510000000000003e-06 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868583+0000", 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8139999999999998e-06 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886746+0000", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018163190999999999 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886808+0000", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1564999999999998e-05 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886889+0000", 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1198000000000004e-05 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1675 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272040267, 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018479580999999998, 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1675 2024-09-21T22:50:27.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868588+0000", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.91e-07 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868589+0000", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.576e-06 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868598+0000", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.551e-06 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868602+0000", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.794e-06 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886929+0000", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018327209000000001 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886992+0000", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3553000000000006e-05 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887067+0000", 2024-09-21T22:50:27.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4606999999999999e-05 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1676 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.272001205, 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018629687999999998, 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1676 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868627+0000", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-07 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868629+0000", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1789999999999998e-06 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868636+0000", 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868642+0000", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3579999999999999e-06 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887104+0000", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018462545 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887176+0000", 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1540999999999998e-05 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887256+0000", 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0141000000000004e-05 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1694 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271918510000001, 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018819487999999999, 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1694 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9700000000000003e-07 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868712+0000", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1619999999999998e-06 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868716+0000", 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6099999999999999e-06 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868720+0000", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887298+0000", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018577750000000001 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887385+0000", 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7077000000000007e-05 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887529+0000", 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014399200000000001 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1695 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271900127, 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01924435, 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1695 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868730+0000", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4830000000000002e-06 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868757+0000", 2024-09-21T22:50:27.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6149999999999999e-05 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868762+0000", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9849999999999997e-06 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887608+0000", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018846471 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887781+0000", 2024-09-21T22:50:27.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017245300000000001 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887972+0000", 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019134100000000001 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1699 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271858526999999, 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019655261, 2024-09-21T22:50:27.185 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1699 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868770+0000", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0699999999999998e-07 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868771+0000", 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9300000000000002e-06 2024-09-21T22:50:27.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868777+0000", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1270000000000002e-06 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868780+0000", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6830000000000001e-06 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888128+0000", 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019347907000000001 2024-09-21T22:50:27.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888265+0000", 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013713700000000001 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888424+0000", 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015907000000000001 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1700 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.27184295, 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019917859999999999, 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1700 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0199999999999998e-07 2024-09-21T22:50:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868786+0000", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4750000000000001e-06 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868793+0000", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868799+0000", 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.596e-06 2024-09-21T22:50:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888513+0000", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019714381999999999 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888606+0000", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2514000000000002e-05 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888703+0000", 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6559999999999997e-05 2024-09-21T22:50:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1701 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271820505999999, 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020220140000000001, 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1701 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868808+0000", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868810+0000", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868816+0000", 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868820+0000", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888804+0000", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019984927 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888894+0000", 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9239000000000004e-05 2024-09-21T22:50:27.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889027+0000", 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133566 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1704 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271803160999999, 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020436185999999999, 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1704 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868825+0000", 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868826+0000", 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.454e-06 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868831+0000", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6009999999999997e-06 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868836+0000", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5230000000000003e-06 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889078+0000", 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020241278000000001 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889176+0000", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8041000000000005e-05 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889261+0000", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4946999999999998e-05 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1705 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:27.202 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271718204000001, 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020613012999999999, 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1705 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868910+0000", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868914+0000", 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9310000000000001e-06 2024-09-21T22:50:27.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868925+0000", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0586e-05 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868933+0000", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9689999999999994e-06 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889307+0000", 2024-09-21T22:50:27.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020373901 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889399+0000", 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2146999999999997e-05 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889522+0000", 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012362500000000001 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1706 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271681083000001, 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021099340000000001, 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1706 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.207 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868950+0000", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9869999999999999e-06 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868959+0000", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9069999999999996e-06 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868965+0000", 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889619+0000", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020653120000000001 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889851+0000", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023190799999999999 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890046+0000", 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000195389 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.210 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1707 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271652962999999, 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022001837, 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1707 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2500000000000005e-07 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868978+0000", 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8949999999999998e-06 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868989+0000", 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1241e-05 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868996+0000", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6499999999999999e-06 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890125+0000", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021128457 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890278+0000", 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015349800000000001 2024-09-21T22:50:27.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890977+0000", 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069847099999999999 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1708 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.271621447999999, 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022287839, 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1708 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869007+0000", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5000000000000003e-07 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869010+0000", 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9469999999999999e-06 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869020+0000", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0316000000000001e-05 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869027+0000", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5939999999999999e-06 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891011+0000", 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021984786999999999 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891138+0000", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012628800000000001 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891294+0000", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015635700000000001 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1807 3.12 3:4e819a64:::rbd_data.111ea6ec99f1.000000000000009f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:27.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.075884295, 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027460622000000001, 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1807 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064747+0000", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7929999999999999e-06 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064763+0000", 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5603999999999998e-05 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064797+0000", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4319999999999997e-05 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064833+0000", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5666e-05 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065001+0000", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016830799999999999 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065362+0000", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036063099999999998 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.070173+0000", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048112459999999999 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092106+0000", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021932953000000002 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092120+0000", 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4318000000000001e-05 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092162+0000", 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1980999999999997e-05 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092204+0000", 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1801999999999997e-05 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3757 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.9200658270000002, 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017800281000000001, 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3757 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:27.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220563+0000", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220568+0000", 2024-09-21T22:50:27.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3299999999999998e-06 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220582+0000", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4047e-05 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220592+0000", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5570000000000001e-06 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230061+0000", 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094694610000000002 2024-09-21T22:50:27.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230174+0000", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011302300000000001 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230809+0000", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063446900000000003 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232849+0000", 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002040395 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238192+0000", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053432089999999998 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238279+0000", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6624000000000003e-05 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238362+0000", 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3295999999999997e-05 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.229 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3758 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.9200139099999998, 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017995779, 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3758 2024-09-21T22:50:27.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220615+0000", 2024-09-21T22:50:27.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2500000000000004e-07 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220620+0000", 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5709999999999997e-06 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220636+0000", 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6030999999999999e-05 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220647+0000", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0411000000000001e-05 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231339+0000", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010691849999999999 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231406+0000", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7846000000000001e-05 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:27.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231663+0000", 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025653299999999998 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.233751+0000", 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020885869999999998 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238457+0000", 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047057230000000002 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238543+0000", 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6191000000000002e-05 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238610+0000", 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6111000000000005e-05 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:27.235 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:50:27.236 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:27.282 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:27.277+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:27.298 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:27.294+0000 7ffa211167c0 -1 Falling back to public interface 2024-09-21T22:50:27.325 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone6 from 0xdd2d930 (overlap 0x47d10c) to 0x0 2024-09-21T22:50:27.339 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:27.334+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:27.415 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:27.415 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:27.415 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:27.415 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:27.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:27.476 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:27.473+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:27.541 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:27.538+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:27.555 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:27.555 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:27.555 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:27.555 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:27.555 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:27.561 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T22:50:27.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:27.597 INFO:teuthology.orchestra.run.smithi094.stdout:1710 writesame 0xe587030 thru 0xe588ca4 data_size 0x1(0x1c75 bytes) 2024-09-21T22:50:27.621 INFO:teuthology.orchestra.run.smithi094.stdout:1711 write 0xee59562 thru 0xee66702 (0xd1a1 bytes) 2024-09-21T22:50:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:1712 clone 8 order 23 su 2097152 sc 7 2024-09-21T22:50:27.641 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:27.637+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:27.724 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:449 3.0 3:04e5c2a8:::rbd_data.111ecbf88771.0000000000000070:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.253543461, 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010416138, 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 449 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:27.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466074+0000", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7240000000000001e-06 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466093+0000", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9432999999999998e-05 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466112+0000", 2024-09-21T22:50:27.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8777e-05 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466121+0000", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8699999999999998e-06 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466198+0000", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7168000000000004e-05 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466348+0000", 2024-09-21T22:50:27.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150359 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.467428+0000", 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010794089999999999 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476341+0000", 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0089129889999999996 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476418+0000", 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6756999999999997e-05 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476450+0000", 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2313e-05 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476487+0000", 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7339000000000003e-05 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:450 3.0 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.253402769999999, 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010348049999999999, 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 450 2024-09-21T22:50:27.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6400000000000002e-07 2024-09-21T22:50:27.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466217+0000", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1169999999999997e-06 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466230+0000", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3653e-05 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466242+0000", 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1479e-05 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472115+0000", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058731429999999999 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472225+0000", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110074 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472818+0000", 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059322200000000002 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476407+0000", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035886849999999999 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476497+0000", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0493000000000002e-05 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476536+0000", 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8464999999999998e-05 2024-09-21T22:50:27.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476560+0000", 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4054999999999999e-05 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:449 3.0 e28/24 ondisk, result = 0)", 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.250267925999999, 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070061350000000001, 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5899999999999995e-07 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469351+0000", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469362+0000", 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0709e-05 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469369+0000", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7839999999999998e-06 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476323+0000", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0069546770000000003 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476333+0000", 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9620000000000005e-06 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476353+0000", 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9544000000000001e-05 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:656 3.0 3:06e36901:::rbd_data.111ecbf88771.00000000000000fd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.996916344000001, 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080847060000000005, 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 656 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:27.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722700+0000", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9970000000000001e-06 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722709+0000", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8729999999999996e-06 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722736+0000", 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6477000000000001e-05 2024-09-21T22:50:27.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722748+0000", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722868+0000", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120562 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.723121+0000", 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025297999999999998 2024-09-21T22:50:27.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.724232+0000", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011103619999999999 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730644+0000", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064117250000000001 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730666+0000", 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2220000000000001e-05 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730726+0000", 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0235000000000002e-05 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730783+0000", 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6885000000000002e-05 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1334 3.19 3:9fa18b7a:::rbd_data.111ea6ec99f1.00000000000001cc:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.92983969, 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085474309999999994, 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1334 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789777+0000", 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1749999999999999e-06 2024-09-21T22:50:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789787+0000", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9380000000000004e-06 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789810+0000", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2864e-05 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789819+0000", 2024-09-21T22:50:27.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4739999999999994e-06 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789889+0000", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9701000000000002e-05 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.790014+0000", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000124595 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.790936+0000", 2024-09-21T22:50:27.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092237099999999995 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798087+0000", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071508090000000002 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798162+0000", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5091000000000003e-05 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798255+0000", 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3528000000000003e-05 2024-09-21T22:50:27.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798322+0000", 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6884999999999994e-05 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1474 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.866343422, 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059933440000000003, 2024-09-21T22:50:27.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1474 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853273+0000", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9800000000000001e-06 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853280+0000", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.516e-06 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853300+0000", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0086999999999999e-05 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853314+0000", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4426999999999999e-05 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.856768+0000", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034537449999999998 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.856974+0000", 2024-09-21T22:50:27.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020624200000000001 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859264+0000", 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022903469999999999 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1506 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.864426460000001, 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053295920000000002, 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1506 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855189+0000", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855195+0000", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855216+0000", 2024-09-21T22:50:27.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0336e-05 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855230+0000", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4163e-05 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859943+0000", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047134539999999997 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860163+0000", 2024-09-21T22:50:27.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022026500000000001 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860518+0000", 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035424399999999998 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1602 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.860630071999999, 2024-09-21T22:50:27.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051835220000000003, 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1602 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858986+0000", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1400000000000001e-06 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859005+0000", 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9714000000000001e-05 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859024+0000", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8377e-05 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859037+0000", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2977e-05 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.863598+0000", 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045609120000000003 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.863829+0000", 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023124199999999999 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.864168+0000", 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00033916000000000003 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1633 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.788 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.859397941999999, 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052612559999999997, 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1633 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:27.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860218+0000", 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3430000000000001e-06 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860223+0000", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9479999999999999e-06 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860241+0000", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7937000000000001e-05 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860254+0000", 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3303e-05 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.864990+0000", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047358749999999996 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865212+0000", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022214500000000001 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865478+0000", 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026570499999999998 2024-09-21T22:50:27.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1634 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.859342527999999, 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.006331957, 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1634 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860273+0000", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860278+0000", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7090000000000001e-06 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860300+0000", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2110999999999999e-05 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860313+0000", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3117e-05 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865563+0000", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052501570000000001 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865781+0000", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217378 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866604+0000", 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082327099999999999 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1665 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.859032924999999, 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063088179999999999, 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1665 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860583+0000", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3289999999999999e-06 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860588+0000", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5400000000000003e-06 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860607+0000", 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8051e-05 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860620+0000", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3668e-05 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866187+0000", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055673019999999997 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866620+0000", 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043221299999999998 2024-09-21T22:50:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866890+0000", 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027071499999999999 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1666 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.858935822999999, 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0069166710000000001, 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1666 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860680+0000", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2550000000000001e-06 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860686+0000", 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4509999999999997e-06 2024-09-21T22:50:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860707+0000", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0937e-05 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860722+0000", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4717e-05 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867049+0000", 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063272010000000002 2024-09-21T22:50:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867268+0000", 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021905899999999999 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867595+0000", 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032705100000000002 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3483 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5258402570000005, 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014021865999999999, 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3483 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193775+0000", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0200000000000001e-07 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193779+0000", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3059999999999999e-06 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193788+0000", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0839999999999998e-06 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193796+0000", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2320000000000004e-06 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193947+0000", 2024-09-21T22:50:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015081400000000001 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194268+0000", 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032190499999999999 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.207796+0000", 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013527723 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3791 3.0 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4936328549999995, 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0068018389999999996, 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3791 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226369+0000", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00038704599999999999 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226383+0000", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3188e-05 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226391+0000", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5960000000000004e-06 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226507+0000", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011558200000000001 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226671+0000", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016398399999999999 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227037+0000", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036616600000000002 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229514+0000", 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002477431 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232671+0000", 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031565019999999998 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232737+0000", 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6427000000000006e-05 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232784+0000", 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6143000000000002e-05 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3792 3.0 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4931966590000005, 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065155550000000001, 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.815 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3792 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:27.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226419+0000", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1399999999999996e-07 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226431+0000", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1836e-05 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226441+0000", 2024-09-21T22:50:27.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0788000000000001e-05 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226448+0000", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5769999999999999e-06 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228354+0000", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001906046 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228532+0000", 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000178479 2024-09-21T22:50:27.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228820+0000", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028807300000000002 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230390+0000", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001569254 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232844+0000", 2024-09-21T22:50:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024540909999999998 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232892+0000", 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8229000000000002e-05 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232933+0000", 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1368000000000001e-05 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3834 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4810442740000003, 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079824639999999999, 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3834 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238573+0000", 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.266e-06 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238793+0000", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220805 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238816+0000", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3167e-05 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238834+0000", 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7050000000000001e-05 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238939+0000", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105586 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.239061+0000", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012182500000000001 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.239332+0000", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000271418 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.244781+0000", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054488289999999997 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246389+0000", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001607863 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246484+0000", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5174999999999996e-05 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246553+0000", 2024-09-21T22:50:27.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8479999999999995e-05 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3835 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4807342929999994, 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010907765, 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3835 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238882+0000", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-06 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238888+0000", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2110000000000003e-06 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238904+0000", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238916+0000", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1532000000000001e-05 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242095+0000", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031794459999999998 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242209+0000", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0001139 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242442+0000", 2024-09-21T22:50:27.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023316100000000001 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.245461+0000", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030189750000000001 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249562+0000", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041011789999999999 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249733+0000", 2024-09-21T22:50:27.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017029499999999999 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249788+0000", 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5464999999999998e-05 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3860 3.19 3:9ca2baa7:::rbd_data.111e7572a6e.0000000000000011:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.4788148200000002, 2024-09-21T22:50:27.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010454101, 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3860 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240801+0000", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1170000000000001e-06 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240807+0000", 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.896e-06 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240824+0000", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7226000000000001e-05 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240858+0000", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3779999999999998e-05 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.244828+0000", 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039701989999999998 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.245002+0000", 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017436 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.251254+0000", 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0062515230000000001 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:4610 3.19 3:9ad11ac6:::rbd_data.111e20a3ef21.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 378183] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.0682139359999994, 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053903379999999997, 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 4610 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651402+0000", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-06 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651603+0000", 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020120000000000001 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651622+0000", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9389999999999999e-05 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651632+0000", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9599999999999995e-06 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651770+0000", 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013722800000000001 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.652144+0000", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037469100000000002 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.652867+0000", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072236399999999997 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.655650+0000", 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027828990000000001 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656607+0000", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095755000000000005 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656708+0000", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100572 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656791+0000", 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3162000000000001e-05 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6028 3.0 3:074660d7:::rbd_data.111ebc806776.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.0826183919999997, 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012713250000000001, 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6028 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636998+0000", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.61e-06 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:27.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637005+0000", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8070000000000002e-06 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637018+0000", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3679999999999999e-05 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637026+0000", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6020000000000002e-06 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:27.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637089+0000", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2994e-05 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637144+0000", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5504999999999998e-05 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.639064+0000", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019201649999999999 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:27.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649635+0000", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010570296999999999 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649647+0000", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2262000000000001e-05 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649678+0000", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1075000000000003e-05 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649709+0000", 2024-09-21T22:50:27.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1253e-05 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:27.846 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:27.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:27.847 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:27.842+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:27.948 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:27.941+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:28.015 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:28.015 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:28.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:28.073 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:28.069+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:28.150 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:28.145+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:28.202 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:28.202 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:28.202 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:28.202 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:28.202 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:28.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:28.261 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:28.257+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:28.310 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:28.360 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:28.357+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:28.373 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone7 from 0xee66703 (overlap 0xdd2d930) to 0x0 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.396 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:726 3.0 e28/24 3:07ca066a:::rbd_data.111ecbf88771.0000000000000134:head v 28'9, mlcod=28'6)", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.596994774000001, 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016183619999999999, 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797023+0000", 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797024+0000", 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0599999999999999e-07 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797030+0000", 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4969999999999998e-06 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797044+0000", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3868e-05 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797052+0000", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1869999999999997e-06 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797110+0000", 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7995000000000003e-05 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.797129+0000", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9344e-05 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798601+0000", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014715419999999999 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.798641+0000", 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0022999999999999e-05 2024-09-21T22:50:28.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1020 3.0 e29/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 29'15, mlcod=28'12)", 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.22858589, 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018427630000000001, 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:28.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165434+0000", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165447+0000", 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3157e-05 2024-09-21T22:50:28.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165474+0000", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6621000000000001e-05 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165487+0000", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3529999999999999e-05 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165644+0000", 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015642 2024-09-21T22:50:28.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165712+0000", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8646000000000006e-05 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167238+0000", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001525804 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167274+0000", 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6329999999999999e-05 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.404 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1030 3.0 e29/24 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head v 29'19, mlcod=29'13)", 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.224482908000001, 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022031799999999999, 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5499999999999999e-07 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169544+0000", 2024-09-21T22:50:28.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9320000000000002e-06 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169551+0000", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3209999999999999e-06 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169557+0000", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5779999999999998e-06 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169973+0000", 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00041665499999999999 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169994+0000", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0679999999999999e-05 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171635+0000", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016405040000000001 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171738+0000", 2024-09-21T22:50:28.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103155 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1349 3.0 e29/24 3:05bd433a:::rbd_data.111ea6ec99f1.00000000000000bb:head v 29'21, mlcod=29'19)", 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.559853662, 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020573869999999999, 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834165+0000", 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6190000000000001e-06 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834177+0000", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.189e-05 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834200+0000", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3110999999999999e-05 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834210+0000", 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0009e-05 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834373+0000", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016245800000000001 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834430+0000", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7402999999999999e-05 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836171+0000", 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017403659999999999 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836221+0000", 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0531000000000001e-05 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1848 3.4 e29/24 3:252aa742:::rbd_data.111ea6ec99f1.00000000000002cf:head v 29'14, mlcod=29'12)", 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.238794108, 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001883619, 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155226+0000", 2024-09-21T22:50:28.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155264+0000", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8418999999999999e-05 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155305+0000", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1029999999999998e-05 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155329+0000", 2024-09-21T22:50:28.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3623000000000001e-05 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155486+0000", 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015655899999999999 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155527+0000", 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1083e-05 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157019+0000", 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001492555 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157107+0000", 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7723e-05 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1868 3.0 e29/24 3:031c5811:::rbd_data.111ea6ec99f1.00000000000001a4:head v 29'22, mlcod=29'20)", 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:28.417 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.209769981999999, 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016467299999999999, 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:28.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184250+0000", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6759999999999999e-06 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184311+0000", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0927000000000001e-05 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184344+0000", 2024-09-21T22:50:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3376e-05 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184360+0000", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5109999999999999e-05 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184459+0000", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8964999999999999e-05 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184487+0000", 2024-09-21T22:50:28.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8092000000000001e-05 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185841+0000", 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013539789999999999 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185894+0000", 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3604999999999999e-05 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2008 3.4 e29/24 3:27f59992:::rbd_data.111ea6ec99f1.000000000000037d:head v 29'16, mlcod=29'14)", 2024-09-21T22:50:28.421 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.000310058, 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016704179999999999, 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:28.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393709+0000", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0729999999999999e-06 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393727+0000", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7227e-05 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393753+0000", 2024-09-21T22:50:28.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6066000000000001e-05 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393769+0000", 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.609e-05 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393939+0000", 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017003600000000001 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.394002+0000", 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3504999999999996e-05 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395320+0000", 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001317599 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395378+0000", 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7822000000000003e-05 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2114 3.9 e29/24 3:91282459:::rbd_data.111ea6ec99f1.00000000000002fb:head v 29'13, mlcod=29'11)", 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.831145812000001, 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001640273, 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:28.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562874+0000", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.03e-06 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562894+0000", 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0752999999999999e-05 2024-09-21T22:50:28.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562914+0000", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9987e-05 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562926+0000", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1153e-05 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563053+0000", 2024-09-21T22:50:28.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127687 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563093+0000", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9646e-05 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564463+0000", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013701080000000001 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564512+0000", 2024-09-21T22:50:28.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8909000000000002e-05 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3136 3.9 e31/24 3:901d0113:::rbd_data.111ea6ec99f1.000000000000000d:head v 31'24, mlcod=30'21)", 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 11.199582965999999, 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021943800000000001, 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194440+0000", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5840000000000002e-06 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194452+0000", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1627999999999999e-05 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194459+0000", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5119999999999997e-06 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194533+0000", 2024-09-21T22:50:28.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4820000000000005e-05 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194575+0000", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1879999999999999e-05 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196580+0000", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020045559999999998 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196629+0000", 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9088000000000002e-05 2024-09-21T22:50:28.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.166790426, 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:50:28.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3792 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'42, mlcod=32'38)", 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.164125737999999, 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001678405, 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229894+0000", 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9010000000000001e-06 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229904+0000", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0808999999999999e-05 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229926+0000", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1778000000000001e-05 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229941+0000", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4831000000000001e-05 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230115+0000", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017410799999999999 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230138+0000", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2714999999999999e-05 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231505+0000", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013669789999999999 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231570+0000", 2024-09-21T22:50:28.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5283999999999994e-05 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3952 3.4 e32/24 3:27093f40:::rbd_data.111e7572a6e.0000000000000013:head v 32'39, mlcod=32'37)", 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.134861954, 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019766520000000002, 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4e-07 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259228+0000", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1829999999999995e-05 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259244+0000", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6028999999999998e-05 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259253+0000", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5730000000000008e-06 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259328+0000", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5209000000000004e-05 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259353+0000", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4626000000000001e-05 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261077+0000", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001724127 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261132+0000", 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5318000000000001e-05 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4769 3.0 e32/24 3:054c8ea6:::rbd_header.111e20a3ef21:head v 32'66, mlcod=32'64)", 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.004635+0000", 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3893823839999992, 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001631941, 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004635+0000", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004635+0000", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004637+0000", 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.34e-06 2024-09-21T22:50:28.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004653+0000", 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5113000000000001e-05 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004685+0000", 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2030000000000003e-05 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004707+0000", 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2050000000000001e-05 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004874+0000", 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016710499999999999 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.004913+0000", 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9569e-05 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.006156+0000", 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012426749999999999 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.006267+0000", 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011105899999999999 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4778 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'49, mlcod=32'47)", 2024-09-21T22:50:28.450 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3775384319999997, 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024572740000000002, 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016481+0000", 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.813e-06 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016608+0000", 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012684600000000001 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016642+0000", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4150000000000003e-05 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016661+0000", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8669999999999999e-05 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016807+0000", 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014669499999999999 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016875+0000", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8163000000000005e-05 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018819+0000", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019438350000000001 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018936+0000", 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117102 2024-09-21T22:50:28.454 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4790 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'50, mlcod=32'48)", 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3604537210000007, 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001951752, 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033566+0000", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.81e-06 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033609+0000", 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3550000000000001e-05 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033639+0000", 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9819999999999999e-05 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033653+0000", 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3675999999999999e-05 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033764+0000", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011095300000000001 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033801+0000", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7412e-05 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035439+0000", 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001638495 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035516+0000", 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6036000000000002e-05 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4796 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'51, mlcod=32'49)", 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3403294829999997, 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001711647, 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053690+0000", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.244e-06 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053705+0000", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4856e-05 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053739+0000", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3543000000000001e-05 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053754+0000", 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5452999999999999e-05 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053901+0000", 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014656900000000001 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053967+0000", 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6602999999999999e-05 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055327+0000", 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013597329999999999 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055400+0000", 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2645999999999995e-05 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4809 3.9 e32/24 3:9625e1c1:::rbd_data.111e20a3ef21.00000000000000fb:head v 32'37, mlcod=32'35)", 2024-09-21T22:50:28.463 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3214935049999994, 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016494910000000001, 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:28.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072526+0000", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.86e-06 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072538+0000", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1960000000000001e-05 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072565+0000", 2024-09-21T22:50:28.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.747e-05 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072584+0000", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8816000000000001e-05 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072721+0000", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136823 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072761+0000", 2024-09-21T22:50:28.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0386000000000003e-05 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.074090+0000", 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013291900000000001 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.074173+0000", 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2985999999999995e-05 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4816 3.0 e32/24 3:048c4bde:::rbd_data.111e20a3ef21.0000000000000083:head v 32'69, mlcod=32'67)", 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3058672920000003, 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001935525, 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088152+0000", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-06 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088201+0000", 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8479999999999997e-05 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088222+0000", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1137000000000001e-05 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088232+0000", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0267e-05 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088310+0000", 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8208999999999996e-05 2024-09-21T22:50:28.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088346+0000", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5376000000000001e-05 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.089996+0000", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016501300000000001 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.090086+0000", 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9975999999999994e-05 2024-09-21T22:50:28.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:6552 3.0 e34/24 3:074660d7:::rbd_data.111ebc806776.0000000000000010:head v 34'96, mlcod=34'94)", 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.335239434, 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021954069999999999, 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058781+0000", 2024-09-21T22:50:28.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4739999999999999e-06 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058829+0000", 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8155999999999998e-05 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058863+0000", 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4139e-05 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058879+0000", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6214000000000001e-05 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059078+0000", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019923799999999999 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059120+0000", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1813000000000001e-05 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060834+0000", 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017137140000000001 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060973+0000", 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013965900000000001 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:8685 3.9 e37/24 3:90954f07:::rbd_data.111e1fb363f5.0000000000000047:head v 37'46, mlcod=36'43)", 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.0602410410000001, 2024-09-21T22:50:28.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021535230000000001, 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:28.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333778+0000", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.658e-06 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333789+0000", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0791000000000001e-05 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333812+0000", 2024-09-21T22:50:28.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2782999999999999e-05 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333824+0000", 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333967+0000", 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014324199999999999 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:28.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.334030+0000", 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2139999999999998e-05 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335821+0000", 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001791579 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335930+0000", 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010884599999999999 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:28.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:28.481 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:28.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:28.482 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:28.482 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:28.483 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:28.477+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:28.506 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:28.506 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:28.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:28.565 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:28.565 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:28.583 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:28.577+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:28.584 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:28.603 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:28.603 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:28.603 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:28.622 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:28.622 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:28.625 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:28.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:28.644 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:28.644 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:28.648 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T22:50:28.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:28.664 INFO:teuthology.orchestra.run.smithi094.stdout:1713 trunc from 0xee66703 to 0x56e3f8d 2024-09-21T22:50:28.683 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:28.678+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:28.705 INFO:teuthology.orchestra.run.smithi094.stdout:1714 read 0x4bd65fa thru 0x4bd8f2a (0x2931 bytes) 2024-09-21T22:50:28.705 INFO:teuthology.orchestra.run.smithi094.stdout:1717 read 0x20353a7 thru 0x203f7da (0xa434 bytes) 2024-09-21T22:50:28.705 INFO:teuthology.orchestra.run.smithi094.stdout:1718 write 0x764d43 thru 0x769b5b (0x4e19 bytes) 2024-09-21T22:50:28.751 INFO:teuthology.orchestra.run.smithi094.stdout:1719 read 0x525e334 thru 0x52622b7 (0x3f84 bytes) 2024-09-21T22:50:28.752 INFO:teuthology.orchestra.run.smithi094.stdout:1720 read 0x440655e thru 0x4415836 (0xf2d9 bytes) 2024-09-21T22:50:28.784 INFO:teuthology.orchestra.run.smithi094.stdout:1721 write 0xcbecef6 thru 0xcbf158b (0x4696 bytes) 2024-09-21T22:50:28.784 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:28.782+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:28.786 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:28.786 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:28.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:1722 trunc from 0xcbf158c to 0x291d3af 2024-09-21T22:50:28.886 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:28.882+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:28.904 INFO:teuthology.orchestra.run.smithi094.stdout:1723 trunc from 0x291d3af to 0x6739c1c 2024-09-21T22:50:28.914 INFO:teuthology.orchestra.run.smithi094.stdout:1724 read 0x461a506 thru 0x46234ab (0x8fa6 bytes) 2024-09-21T22:50:28.914 INFO:teuthology.orchestra.run.smithi094.stdout:1725 read 0x1dd9c4e thru 0x1ddc123 (0x24d6 bytes) 2024-09-21T22:50:28.915 INFO:teuthology.orchestra.run.smithi094.stdout:1726 write 0x5b93e66 thru 0x5b9c9bb (0x8b56 bytes) 2024-09-21T22:50:28.925 INFO:teuthology.orchestra.run.smithi094.stdout:1727 trunc from 0x6739c1c to 0x8335ac2 2024-09-21T22:50:28.948 INFO:teuthology.orchestra.run.smithi094.stdout:1728 trunc from 0x8335ac2 to 0xe3e4940 2024-09-21T22:50:28.952 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:28.953 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:28.961 INFO:teuthology.orchestra.run.smithi094.stdout:1730 punch from 0xc034bc8 to 0xc03a12d, (0x5565 bytes) 2024-09-21T22:50:28.961 INFO:teuthology.orchestra.run.smithi094.stdout:1731 read 0x3b6f612 thru 0x3b7c0ae (0xca9d bytes) 2024-09-21T22:50:28.961 INFO:teuthology.orchestra.run.smithi094.stdout:1732 write 0x3eb49af thru 0x3eb8d9c (0x43ee bytes) 2024-09-21T22:50:28.964 INFO:teuthology.orchestra.run.smithi094.stdout:1733 write 0x3d5c17f thru 0x3d689bd (0xc83f bytes) 2024-09-21T22:50:28.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:28.970 INFO:teuthology.orchestra.run.smithi094.stdout:1734 read 0x6f0b497 thru 0x6f12414 (0x6f7e bytes) 2024-09-21T22:50:28.970 INFO:teuthology.orchestra.run.smithi094.stdout:1735 punch from 0x4df5f66 to 0x4e05014, (0xf0ae bytes) 2024-09-21T22:50:28.970 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.001 INFO:teuthology.orchestra.run.smithi094.stdout:1738 trunc from 0xe3e4940 to 0x836c62 2024-09-21T22:50:29.017 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:29.014+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:29.071 INFO:teuthology.orchestra.run.smithi094.stdout:1739 write 0xb419be0 thru 0xb422792 (0x8bb3 bytes) 2024-09-21T22:50:29.078 INFO:teuthology.orchestra.run.smithi094.stdout:1740 read 0x2c919bc thru 0x2c9f230 (0xd875 bytes) 2024-09-21T22:50:29.079 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.086 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:29.081+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:29.089 INFO:teuthology.orchestra.run.smithi094.stdout:1742 trunc from 0xb422793 to 0x731614b 2024-09-21T22:50:29.105 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:29.105 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:29.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:29.124 INFO:teuthology.orchestra.run.smithi094.stdout:1746 trunc from 0x731614b to 0xa9245ff 2024-09-21T22:50:29.135 INFO:teuthology.orchestra.run.smithi094.stdout:1748 read 0x82b79f9 thru 0x82ba8dc (0x2ee4 bytes) 2024-09-21T22:50:29.135 INFO:teuthology.orchestra.run.smithi094.stdout:1749 writesame 0x70cfd55 thru 0x70dc5e7 data_size 0x1(0xc893 bytes) 2024-09-21T22:50:29.186 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:29.181+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1751 read 0x53191be thru 0x53237e3 (0xa626 bytes) 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1752 read 0xd3bd47 thru 0xd43f99 (0x8253 bytes) 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1754 punch from 0x49dc084 to 0x49e9246, (0xd1c2 bytes) 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1755 read 0x4c57fb5 thru 0x4c5d5e5 (0x5631 bytes) 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1756 punch from 0x8c6ebeb to 0x8c6f5f3, (0xa08 bytes) 2024-09-21T22:50:29.207 INFO:teuthology.orchestra.run.smithi094.stdout:1757 write 0x78a1708 thru 0x78b1453 (0xfd4c bytes) 2024-09-21T22:50:29.214 INFO:teuthology.orchestra.run.smithi094.stdout:1759 write 0x2460f46 thru 0x246a26b (0x9326 bytes) 2024-09-21T22:50:29.218 INFO:teuthology.orchestra.run.smithi094.stdout:1761 punch from 0x1734b43 to 0x173693f, (0x1dfc bytes) 2024-09-21T22:50:29.218 INFO:teuthology.orchestra.run.smithi094.stdout:1764 read 0x770ce06 thru 0x77133bf (0x65ba bytes) 2024-09-21T22:50:29.218 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.218 INFO:teuthology.orchestra.run.smithi094.stdout:1769 read 0x47e9fa5 thru 0x47f3ac8 (0x9b24 bytes) 2024-09-21T22:50:29.218 INFO:teuthology.orchestra.run.smithi094.stdout:1770 flatten 2024-09-21T22:50:29.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:50:29.242 INFO:teuthology.orchestra.run.smithi094.stdout:1773 trunc from 0xa9245ff to 0xe358301 2024-09-21T22:50:29.255 INFO:teuthology.orchestra.run.smithi094.stdout:1774 read 0x9e2ec05 thru 0x9e301be (0x15ba bytes) 2024-09-21T22:50:29.255 INFO:teuthology.orchestra.run.smithi094.stdout:1775 writesame 0x592469 thru 0x5a0ea7 data_size 0x9(0xea3f bytes) 2024-09-21T22:50:29.261 INFO:teuthology.orchestra.run.smithi094.stdout:1776 read 0xc5abc55 thru 0xc5ad1ad (0x1559 bytes) 2024-09-21T22:50:29.261 INFO:teuthology.orchestra.run.smithi094.stdout:1779 write 0x594f26b thru 0x5952783 (0x3519 bytes) 2024-09-21T22:50:29.266 INFO:teuthology.orchestra.run.smithi094.stdout:1780 punch from 0x6a440a to 0x6b0121, (0xbd17 bytes) 2024-09-21T22:50:29.266 INFO:teuthology.orchestra.run.smithi094.stdout:1781 write 0x2a674b3 thru 0x2a69319 (0x1e67 bytes) 2024-09-21T22:50:29.268 INFO:teuthology.orchestra.run.smithi094.stdout:1783 read 0xa9da3f1 thru 0xa9e3bf6 (0x9806 bytes) 2024-09-21T22:50:29.269 INFO:teuthology.orchestra.run.smithi094.stdout:1784 write 0xa40249 thru 0xa449dc (0x4794 bytes) 2024-09-21T22:50:29.274 INFO:teuthology.orchestra.run.smithi094.stdout:1785 writesame 0x8102181 thru 0x810e13a data_size 0x2(0xbfba bytes) 2024-09-21T22:50:29.284 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:29.284 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:29.287 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:29.282+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:29.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:1787 punch from 0x1299417 to 0x12a6a50, (0xd639 bytes) 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:1796 read 0x61e8268 thru 0x61ebb96 (0x392f bytes) 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.309 INFO:teuthology.orchestra.run.smithi094.stdout:1801 punch from 0xe0aa277 to 0xe0b3e4a, (0x9bd3 bytes) 2024-09-21T22:50:29.332 INFO:teuthology.orchestra.run.smithi094.stdout:1803 trunc from 0xe358301 to 0x38e4443 2024-09-21T22:50:29.361 INFO:teuthology.orchestra.run.smithi094.stdout:1804 punch from 0x21738d5 to 0x217fc26, (0xc351 bytes) 2024-09-21T22:50:29.361 INFO:teuthology.orchestra.run.smithi094.stdout:1805 write 0x34a47b7 thru 0x34a6b9a (0x23e4 bytes) 2024-09-21T22:50:29.363 INFO:teuthology.orchestra.run.smithi094.stdout:1806 read 0x34e823d thru 0x34f651e (0xe2e2 bytes) 2024-09-21T22:50:29.365 INFO:teuthology.orchestra.run.smithi094.stdout:1807 trunc from 0x38e4443 to 0x3074840 2024-09-21T22:50:29.378 INFO:teuthology.orchestra.run.smithi094.stdout:1808 read 0x1b63b04 thru 0x1b738e8 (0xfde5 bytes) 2024-09-21T22:50:29.388 INFO:teuthology.orchestra.run.smithi094.stdout:1809 writesame 0x462c158 thru 0x4635efa data_size 0x7(0x9da3 bytes) 2024-09-21T22:50:29.388 INFO:teuthology.orchestra.run.smithi094.stdout:1812 punch from 0x2793858 to 0x27a29b7, (0xf15f bytes) 2024-09-21T22:50:29.388 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.388 INFO:teuthology.orchestra.run.smithi094.stdout:1814 punch from 0x2f8cc24 to 0x2f95448, (0x8824 bytes) 2024-09-21T22:50:29.388 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:29.385+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:29.399 INFO:teuthology.orchestra.run.smithi094.stdout:1815 write 0x6f2316e thru 0x6f28650 (0x54e3 bytes) 2024-09-21T22:50:29.417 INFO:teuthology.orchestra.run.smithi094.stdout:1816 write 0xacbe4dd thru 0xacc7c9d (0x97c1 bytes) 2024-09-21T22:50:29.422 INFO:teuthology.orchestra.run.smithi094.stdout:1818 write 0x344f89f thru 0x34542c3 (0x4a25 bytes) 2024-09-21T22:50:29.425 INFO:teuthology.orchestra.run.smithi094.stdout:1820 write 0x9468dcc thru 0x9475b66 (0xcd9b bytes) 2024-09-21T22:50:29.441 INFO:teuthology.orchestra.run.smithi094.stdout:1822 writesame 0xb133183 thru 0xb140e56 data_size 0x2a(0xdcd4 bytes) 2024-09-21T22:50:29.441 INFO:teuthology.orchestra.run.smithi094.stdout:1823 writesame 0x884a2fd thru 0x884d0f2 data_size 0x35(0x2df6 bytes) 2024-09-21T22:50:29.447 INFO:teuthology.orchestra.run.smithi094.stdout:1825 punch from 0x9f72791 to 0x9f7c609, (0x9e78 bytes) 2024-09-21T22:50:29.447 INFO:teuthology.orchestra.run.smithi094.stdout:1826 write 0x95bc3c7 thru 0x95c0436 (0x4070 bytes) 2024-09-21T22:50:29.449 INFO:teuthology.orchestra.run.smithi094.stdout:1827 read 0x251ec7f thru 0x2527007 (0x8389 bytes) 2024-09-21T22:50:29.450 INFO:teuthology.orchestra.run.smithi094.stdout:1828 write 0x41944aa thru 0x419c1e3 (0x7d3a bytes) 2024-09-21T22:50:29.456 INFO:teuthology.orchestra.run.smithi094.stdout:1829 writesame 0x56f0a42 thru 0x56f811a data_size 0x5(0x76d9 bytes) 2024-09-21T22:50:29.465 INFO:teuthology.orchestra.run.smithi094.stdout:1830 read 0x16799cb thru 0x1682188 (0x87be bytes) 2024-09-21T22:50:29.466 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.466 INFO:teuthology.orchestra.run.smithi094.stdout:1832 punch from 0xfe8d89 to 0xfee2fc, (0x5573 bytes) 2024-09-21T22:50:29.466 INFO:teuthology.orchestra.run.smithi094.stdout:1833 read 0x899e565 thru 0x89ab8e0 (0xd37c bytes) 2024-09-21T22:50:29.467 INFO:teuthology.orchestra.run.smithi094.stdout:1834 writesame 0x80ab9ef thru 0x80b6f8d data_size 0x11(0xb59f bytes) 2024-09-21T22:50:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:29.489 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:29.486+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:29.491 INFO:teuthology.orchestra.run.smithi094.stdout:1835 writesame 0xdae171a thru 0xdaf029d data_size 0x2(0xeb84 bytes) 2024-09-21T22:50:29.491 INFO:teuthology.orchestra.run.smithi094.stdout:1836 write 0xc98c9e7 thru 0xc99b796 (0xedb0 bytes) 2024-09-21T22:50:29.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:29.494 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.495 INFO:teuthology.orchestra.run.smithi094.stdout:1838 trunc from 0xdaf029e to 0xcf62954 2024-09-21T22:50:29.506 INFO:teuthology.orchestra.run.smithi094.stdout:1840 punch from 0xa5691f5 to 0xa57867a, (0xf485 bytes) 2024-09-21T22:50:29.506 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.516 INFO:teuthology.orchestra.run.smithi094.stdout:1842 trunc from 0xcf62954 to 0x851765c 2024-09-21T22:50:29.535 INFO:teuthology.orchestra.run.smithi094.stdout:1845 write 0x14c6a thru 0x1648a (0x1821 bytes) 2024-09-21T22:50:29.538 INFO:teuthology.orchestra.run.smithi094.stdout:1847 read 0x1605e73 thru 0x1611468 (0xb5f6 bytes) 2024-09-21T22:50:29.548 INFO:teuthology.orchestra.run.smithi094.stdout:1848 trunc from 0x851765c to 0xc7686e1 2024-09-21T22:50:29.559 INFO:teuthology.orchestra.run.smithi094.stdout:1849 read 0x44349b5 thru 0x443c639 (0x7c85 bytes) 2024-09-21T22:50:29.560 INFO:teuthology.orchestra.run.smithi094.stdout:1851 trunc from 0xc7686e1 to 0xc1b6129 2024-09-21T22:50:29.570 INFO:teuthology.orchestra.run.smithi094.stdout:1853 punch from 0x2273da9 to 0x2282e9a, (0xf0f1 bytes) 2024-09-21T22:50:29.570 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.570 INFO:teuthology.orchestra.run.smithi094.stdout:1855 read 0x739850b thru 0x739ec60 (0x6756 bytes) 2024-09-21T22:50:29.570 INFO:teuthology.orchestra.run.smithi094.stdout:1857 read 0x63fdcfa thru 0x6406712 (0x8a19 bytes) 2024-09-21T22:50:29.579 INFO:teuthology.orchestra.run.smithi094.stdout:1858 trunc from 0xc1b6129 to 0x8dfd45e 2024-09-21T22:50:29.589 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:29.586+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:29.590 INFO:teuthology.orchestra.run.smithi094.stdout:1860 writesame 0x1a6924f thru 0x1a71da5 data_size 0x1(0x8b57 bytes) 2024-09-21T22:50:29.612 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.612 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.612 INFO:teuthology.orchestra.run.smithi094.stdout:1864 read 0x22604e4 thru 0x2260684 (0x1a1 bytes) 2024-09-21T22:50:29.626 INFO:teuthology.orchestra.run.smithi094.stdout:1865 writesame 0xab2a5f0 thru 0xab3291c data_size 0x1(0x832d bytes) 2024-09-21T22:50:29.626 INFO:teuthology.orchestra.run.smithi094.stdout:1866 read 0x51c5c16 thru 0x51d0dd8 (0xb1c3 bytes) 2024-09-21T22:50:29.645 INFO:teuthology.orchestra.run.smithi094.stdout:1869 trunc from 0xab3291d to 0x1d1f520 2024-09-21T22:50:29.668 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:29.668 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:29.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:29.697 INFO:teuthology.orchestra.run.smithi094.stdout:1870 writesame 0x3cb0363 thru 0x3cb990e data_size 0x1f(0x95ac bytes) 2024-09-21T22:50:29.697 INFO:teuthology.orchestra.run.smithi094.stdout:1871 read 0x197dfd5 thru 0x197ff20 (0x1f4c bytes) 2024-09-21T22:50:29.725 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:29.722+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:29.731 INFO:teuthology.orchestra.run.smithi094.stdout:1872 write 0xda114bf thru 0xda142b9 (0x2dfb bytes) 2024-09-21T22:50:29.738 INFO:teuthology.orchestra.run.smithi094.stdout:1878 punch from 0x308786c to 0x3096e1b, (0xf5af bytes) 2024-09-21T22:50:29.738 INFO:teuthology.orchestra.run.smithi094.stdout:1879 read 0x469e6ba thru 0x46acd77 (0xe6be bytes) 2024-09-21T22:50:29.760 INFO:teuthology.orchestra.run.smithi094.stdout:1880 trunc from 0xda142ba to 0x4509623 2024-09-21T22:50:29.778 INFO:teuthology.orchestra.run.smithi094.stdout:1882 read 0x1078324 thru 0x10881c1 (0xfe9e bytes) 2024-09-21T22:50:29.791 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:29.786+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:29.793 INFO:teuthology.orchestra.run.smithi094.stdout:1883 write 0x657dbf1 thru 0x658014d (0x255d bytes) 2024-09-21T22:50:29.807 INFO:teuthology.orchestra.run.smithi094.stdout:1884 trunc from 0x658014e to 0x91ba0e1 2024-09-21T22:50:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:1885 writesame 0x573c601 thru 0x573f43a data_size 0x2(0x2e3a bytes) 2024-09-21T22:50:29.841 INFO:teuthology.orchestra.run.smithi094.stdout:1886 writesame 0x94ad18a thru 0x94bc2c4 data_size 0x45(0xf13b bytes) 2024-09-21T22:50:29.841 INFO:teuthology.orchestra.run.smithi094.stdout:1888 write 0x2b79759 thru 0x2b85349 (0xbbf1 bytes) 2024-09-21T22:50:29.845 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.845 INFO:teuthology.orchestra.run.smithi094.stdout:1892 writesame 0x5d9cdc thru 0x5e6970 data_size 0x1(0xcc95 bytes) 2024-09-21T22:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:29.854 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:29.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:29.892 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:29.885+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:29.911 INFO:teuthology.orchestra.run.smithi094.stdout:1893 punch from 0x14b2ba9 to 0x14bf6a2, (0xcaf9 bytes) 2024-09-21T22:50:29.911 INFO:teuthology.orchestra.run.smithi094.stdout:1894 punch from 0x86181c4 to 0x8623327, (0xb163 bytes) 2024-09-21T22:50:29.912 INFO:teuthology.orchestra.run.smithi094.stdout:1895 punch from 0x3bb883 to 0x3c268e, (0x6e0b bytes) 2024-09-21T22:50:29.912 INFO:teuthology.orchestra.run.smithi094.stdout:1897 punch from 0x3f7e523 to 0x3f85951, (0x742e bytes) 2024-09-21T22:50:29.912 INFO:teuthology.orchestra.run.smithi094.stdout:1898 read 0x5e7e5f9 thru 0x5e822f2 (0x3cfa bytes) 2024-09-21T22:50:29.912 INFO:teuthology.orchestra.run.smithi094.stdout:1899 writesame 0x5714a8d thru 0x57189f8 data_size 0x2c(0x3f6c bytes) 2024-09-21T22:50:29.915 INFO:teuthology.orchestra.run.smithi094.stdout:1902 write 0x4434ed5 thru 0x44350e6 (0x212 bytes) 2024-09-21T22:50:29.920 INFO:teuthology.orchestra.run.smithi094.stdout:1903 write 0x8be6202 thru 0x8becb06 (0x6905 bytes) 2024-09-21T22:50:29.926 INFO:teuthology.orchestra.run.smithi094.stdout:1904 write 0x11b5090 thru 0x11c0dce (0xbd3f bytes) 2024-09-21T22:50:29.943 INFO:teuthology.orchestra.run.smithi094.stdout:1905 write 0xae5a747 thru 0xae62ed9 (0x8793 bytes) 2024-09-21T22:50:29.949 INFO:teuthology.orchestra.run.smithi094.stdout:1906 read 0x6e9d11f thru 0x6ea07d9 (0x36bb bytes) 2024-09-21T22:50:29.950 INFO:teuthology.orchestra.run.smithi094.stdout:1907 read 0x1668843 thru 0x1672f4f (0xa70d bytes) 2024-09-21T22:50:29.966 INFO:teuthology.orchestra.run.smithi094.stdout:1908 write 0xd315315 thru 0xd31ddf1 (0x8add bytes) 2024-09-21T22:50:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:1909 read 0xc7c6cb thru 0xc89043 (0xc979 bytes) 2024-09-21T22:50:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:1911 read 0x261c446 thru 0x262b51e (0xf0d9 bytes) 2024-09-21T22:50:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:1912 punch from 0xb6fda85 to 0xb70bb9c, (0xe117 bytes) 2024-09-21T22:50:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:29.984 INFO:teuthology.orchestra.run.smithi094.stdout:1916 trunc from 0xd31ddf2 to 0x8f0cad0 2024-09-21T22:50:29.993 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:29.989+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:30.008 INFO:teuthology.orchestra.run.smithi094.stdout:1917 trunc from 0x8f0cad0 to 0x840ec3b 2024-09-21T22:50:30.023 INFO:teuthology.orchestra.run.smithi094.stdout:1918 read 0x120b740 thru 0x120ec8a (0x354b bytes) 2024-09-21T22:50:30.024 INFO:teuthology.orchestra.run.smithi094.stdout:1919 read 0x1bac663 thru 0x1bb5b5c (0x94fa bytes) 2024-09-21T22:50:30.025 INFO:teuthology.orchestra.run.smithi094.stdout:1921 read 0x34522e1 thru 0x345ab7f (0x889f bytes) 2024-09-21T22:50:30.025 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.026 INFO:teuthology.orchestra.run.smithi094.stdout:1923 write 0x71c9b9b thru 0x71cb2a3 (0x1709 bytes) 2024-09-21T22:50:30.032 INFO:teuthology.orchestra.run.smithi094.stdout:1924 read 0x1d7136e thru 0x1d75dc4 (0x4a57 bytes) 2024-09-21T22:50:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:30.052 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:30.054 INFO:teuthology.orchestra.run.smithi094.stdout:1925 writesame 0xb7a3b65 thru 0xb7b05b8 data_size 0x2e(0xca54 bytes) 2024-09-21T22:50:30.054 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:1928 trunc from 0xb7b05b9 to 0x7f86058 2024-09-21T22:50:30.076 INFO:teuthology.orchestra.run.smithi094.stdout:1929 writesame 0x702f53c thru 0x703a892 data_size 0x1(0xb357 bytes) 2024-09-21T22:50:30.097 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:30.093+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:30.135 INFO:teuthology.orchestra.run.smithi094.stdout:1931 writesame 0x3277dc0 thru 0x32780c7 data_size 0x4(0x308 bytes) 2024-09-21T22:50:30.139 INFO:teuthology.orchestra.run.smithi094.stdout:1933 writesame 0x4aa6099 thru 0x4aac83a data_size 0x23(0x67a2 bytes) 2024-09-21T22:50:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:50:30.170 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:30.170 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:1934 write 0xb2411ec thru 0xb2472da (0x60ef bytes) 2024-09-21T22:50:30.179 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.179 INFO:teuthology.orchestra.run.smithi094.stdout:1936 write 0x27a316 thru 0x283000 (0x8ceb bytes) 2024-09-21T22:50:30.182 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.182 INFO:teuthology.orchestra.run.smithi094.stdout:1938 write 0x516022e thru 0x516d0f2 (0xcec5 bytes) 2024-09-21T22:50:30.185 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:30.186 INFO:teuthology.orchestra.run.smithi094.stdout:1939 write 0x3fe3ea8 thru 0x3feedf2 (0xaf4b bytes) 2024-09-21T22:50:30.189 INFO:teuthology.orchestra.run.smithi094.stdout:1940 writesame 0x934442c thru 0x934bd2f data_size 0xa(0x7904 bytes) 2024-09-21T22:50:30.199 INFO:teuthology.orchestra.run.smithi094.stdout:1942 trunc from 0xb2472db to 0x8eceac6 2024-09-21T22:50:30.215 INFO:teuthology.orchestra.run.smithi094.stdout:1943 read 0x16952c3 thru 0x169a4c8 (0x5206 bytes) 2024-09-21T22:50:30.216 INFO:teuthology.orchestra.run.smithi094.stdout:1944 writesame 0x114985b thru 0x1155e0d data_size 0x2b(0xc5b3 bytes) 2024-09-21T22:50:30.229 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:30.225+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:30.236 INFO:teuthology.orchestra.run.smithi094.stdout:1946 write 0xc184848 thru 0xc192c74 (0xe42d bytes) 2024-09-21T22:50:30.242 INFO:teuthology.orchestra.run.smithi094.stdout:1947 write 0x2309207 thru 0x2317673 (0xe46d bytes) 2024-09-21T22:50:30.245 INFO:teuthology.orchestra.run.smithi094.stdout:1948 read 0x1b96762 thru 0x1b998b2 (0x3151 bytes) 2024-09-21T22:50:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:1952 read 0xbd4fb94 thru 0xbd5d5d3 (0xda40 bytes) 2024-09-21T22:50:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:1953 punch from 0xae21f78 to 0xae27d18, (0x5da0 bytes) 2024-09-21T22:50:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:1955 read 0x47ca6c6 thru 0x47cf3c5 (0x4d00 bytes) 2024-09-21T22:50:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:1956 writesame 0x39e69d5 thru 0x39ed57e data_size 0x1(0x6baa bytes) 2024-09-21T22:50:30.268 INFO:teuthology.orchestra.run.smithi094.stdout:1957 writesame 0x48b23c3 thru 0x48b849b data_size 0x1(0x60d9 bytes) 2024-09-21T22:50:30.294 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:30.289+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:30.297 INFO:teuthology.orchestra.run.smithi094.stdout:1958 trunc from 0xc192c75 to 0xb3f4fce 2024-09-21T22:50:30.310 INFO:teuthology.orchestra.run.smithi094.stdout:1959 writesame 0x21550cb thru 0x2164671 data_size 0x1(0xf5a7 bytes) 2024-09-21T22:50:30.353 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:30.353 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:30.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:30.376 INFO:teuthology.orchestra.run.smithi094.stdout:1960 writesame 0x9b5e0f1 thru 0x9b63033 data_size 0x1(0x4f43 bytes) 2024-09-21T22:50:30.413 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:30.409+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:30.422 INFO:teuthology.orchestra.run.smithi094.stdout:1962 write 0xe7b5ad1 thru 0xe7bae51 (0x5381 bytes) 2024-09-21T22:50:30.428 INFO:teuthology.orchestra.run.smithi094.stdout:1963 punch from 0x1d7b6b to 0x1e144c, (0x98e1 bytes) 2024-09-21T22:50:30.434 INFO:teuthology.orchestra.run.smithi094.stdout:1964 trunc from 0xe7bae52 to 0xc5026b1 2024-09-21T22:50:30.453 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.453 INFO:teuthology.orchestra.run.smithi094.stdout:1966 read 0x118140a thru 0x118f9e3 (0xe5da bytes) 2024-09-21T22:50:30.454 INFO:teuthology.orchestra.run.smithi094.stdout:1968 read 0x110410f thru 0x1112ce0 (0xebd2 bytes) 2024-09-21T22:50:30.455 INFO:teuthology.orchestra.run.smithi094.stdout:1969 read 0x5936131 thru 0x593e37b (0x824b bytes) 2024-09-21T22:50:30.456 INFO:teuthology.orchestra.run.smithi094.stdout:1970 read 0x34624c9 thru 0x346fb61 (0xd699 bytes) 2024-09-21T22:50:30.456 INFO:teuthology.orchestra.run.smithi094.stdout:1971 read 0x59146e9 thru 0x5922408 (0xdd20 bytes) 2024-09-21T22:50:30.457 INFO:teuthology.orchestra.run.smithi094.stdout:1973 writesame 0x3f04028 thru 0x3f08908 data_size 0x9(0x48e1 bytes) 2024-09-21T22:50:30.462 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.462 INFO:teuthology.orchestra.run.smithi094.stdout:1975 read 0x3271497 thru 0x327692f (0x5499 bytes) 2024-09-21T22:50:30.463 INFO:teuthology.orchestra.run.smithi094.stdout:1976 read 0x79d9980 thru 0x79e5216 (0xb897 bytes) 2024-09-21T22:50:30.463 INFO:teuthology.orchestra.run.smithi094.stdout:1977 punch from 0xa26c8b0 to 0xa27bfaf, (0xf6ff bytes) 2024-09-21T22:50:30.463 INFO:teuthology.orchestra.run.smithi094.stdout:1979 write 0x4fc037f thru 0x4fc1480 (0x1102 bytes) 2024-09-21T22:50:30.469 INFO:teuthology.orchestra.run.smithi094.stdout:1981 read 0x433fd4f thru 0x4341806 (0x1ab8 bytes) 2024-09-21T22:50:30.492 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:30.492 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:30.494 INFO:teuthology.orchestra.run.smithi094.stdout:1984 trunc from 0xc5026b1 to 0x36f50a3 2024-09-21T22:50:30.495 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:30.489+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:30.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:30.521 INFO:teuthology.orchestra.run.smithi094.stdout:1986 punch from 0x6c0117 to 0x6cb134, (0xb01d bytes) 2024-09-21T22:50:30.526 INFO:teuthology.orchestra.run.smithi094.stdout:1988 trunc from 0x36f50a3 to 0xd0af93 2024-09-21T22:50:30.543 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.543 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:30.556 INFO:teuthology.orchestra.run.smithi094.stdout:1992 write 0x2d8459e thru 0x2d8eeda (0xa93d bytes) 2024-09-21T22:50:30.558 INFO:teuthology.orchestra.run.smithi094.stdout:1993 read 0x1bf7ad8 thru 0x1c06daa (0xf2d3 bytes) 2024-09-21T22:50:30.559 INFO:teuthology.orchestra.run.smithi094.stdout:1994 read 0x2422a02 thru 0x24305e6 (0xdbe5 bytes) 2024-09-21T22:50:30.560 INFO:teuthology.orchestra.run.smithi094.stdout:1995 punch from 0x26a4749 to 0x26a78e5, (0x319c bytes) 2024-09-21T22:50:30.575 INFO:teuthology.orchestra.run.smithi094.stdout:1996 writesame 0x51f5926 thru 0x5204f4f data_size 0x51(0xf62a bytes) 2024-09-21T22:50:30.575 INFO:teuthology.orchestra.run.smithi094.stdout:1997 punch from 0x5115b84 to 0x5117b22, (0x1f9e bytes) 2024-09-21T22:50:30.575 INFO:teuthology.orchestra.run.smithi094.stdout:1998 read 0x2cb54f thru 0x2cd1b9 (0x1c6b bytes) 2024-09-21T22:50:30.576 INFO:teuthology.orchestra.run.smithi094.stdout:1999 writesame 0x3fd1f31 thru 0x3fdedbd data_size 0x79(0xce8d bytes) 2024-09-21T22:50:30.582 INFO:teuthology.orchestra.run.smithi094.stdout:2002 read 0x493eaf6 thru 0x4944914 (0x5e1f bytes) 2024-09-21T22:50:30.582 INFO:teuthology.orchestra.run.smithi094.stdout:2003 writesame 0x50d856d thru 0x50e78e2 data_size 0xb(0xf376 bytes) 2024-09-21T22:50:30.596 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:30.589+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:30.603 INFO:teuthology.orchestra.run.smithi094.stdout:2007 trunc from 0x5204f50 to 0xa3e053a 2024-09-21T22:50:30.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:30.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:30.638 INFO:teuthology.orchestra.run.smithi094.stdout:2010 write 0xe4f0431 thru 0xe4f1138 (0xd08 bytes) 2024-09-21T22:50:30.643 INFO:teuthology.orchestra.run.smithi094.stdout:2011 read 0x9cad3df thru 0x9cb0334 (0x2f56 bytes) 2024-09-21T22:50:30.643 INFO:teuthology.orchestra.run.smithi094.stdout:2012 write 0x350be83 thru 0x351273f (0x68bd bytes) 2024-09-21T22:50:30.645 INFO:teuthology.orchestra.run.smithi094.stdout:2013 punch from 0xe56046 to 0xe5e9ef, (0x89a9 bytes) 2024-09-21T22:50:30.645 INFO:teuthology.orchestra.run.smithi094.stdout:2015 read 0x2b37b85 thru 0x2b42571 (0xa9ed bytes) 2024-09-21T22:50:30.649 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:30.649 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:30.668 INFO:teuthology.orchestra.run.smithi094.stdout:2016 trunc from 0xe4f1139 to 0x834dec 2024-09-21T22:50:30.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:2023 trunc from 0x834dec to 0x6e4a3f3 2024-09-21T22:50:30.706 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:30.701+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:30.731 INFO:teuthology.orchestra.run.smithi094.stdout:2024 writesame 0x789e8aa thru 0x78a5bf6 data_size 0x1(0x734d bytes) 2024-09-21T22:50:30.731 INFO:teuthology.orchestra.run.smithi094.stdout:2025 read 0x56854b8 thru 0x56947b4 (0xf2fd bytes) 2024-09-21T22:50:30.741 INFO:teuthology.orchestra.run.smithi099.stderr:no valid command found; 10 closest matches: 2024-09-21T22:50:30.741 INFO:teuthology.orchestra.run.smithi099.stderr:0 2024-09-21T22:50:30.741 INFO:teuthology.orchestra.run.smithi099.stderr:1 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:2 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:abort 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:assert 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:bluestore allocator dump block 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:bluestore allocator fragmentation block 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:bluestore allocator score block 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:config diff 2024-09-21T22:50:30.742 INFO:teuthology.orchestra.run.smithi099.stderr:admin_socket: invalid command 2024-09-21T22:50:30.744 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T22:50:30.744 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-21T22:50:30.749 INFO:teuthology.orchestra.run.smithi094.stdout:2027 write 0xd5af7ef thru 0xd5b2245 (0x2a57 bytes) 2024-09-21T22:50:30.754 INFO:teuthology.orchestra.run.smithi094.stdout:2028 clone 9 order 24 su 4194304 sc 8 2024-09-21T22:50:30.780 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:30.781 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:30.797 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:30.793+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:30.797 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:30.797 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:30.814 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:30.814 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:30.830 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:30.832 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:30.832 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:30.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:30.851 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:30.851 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:30.870 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:30.871 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:30.871 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-21T22:50:30.897 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:30.893+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:30.907 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:30.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:50:30.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1667 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107676905000002, 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017652939999999999, 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1667 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868500+0000", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868501+0000", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868503+0000", 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4090000000000001e-06 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868510+0000", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3799999999999996e-06 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868514+0000", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0280000000000002e-06 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886013+0000", 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017498841000000001 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886073+0000", 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0015000000000001e-05 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886153+0000", 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9800000000000002e-05 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1668 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107657377999999, 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017819267999999999, 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1668 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868520+0000", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2599999999999998e-07 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868522+0000", 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7260000000000001e-06 2024-09-21T22:50:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868527+0000", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8010000000000003e-06 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868530+0000", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886193+0000", 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017663239000000001 2024-09-21T22:50:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886254+0000", 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0977000000000002e-05 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886339+0000", 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4622000000000004e-05 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.986 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1672 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107642455000001, 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017989136999999999, 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1672 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:30.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868535+0000", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.96e-07 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868536+0000", 2024-09-21T22:50:30.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.517e-06 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868541+0000", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5059999999999997e-06 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868545+0000", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9009999999999996e-06 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886380+0000", 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017834843 2024-09-21T22:50:30.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886443+0000", 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3443999999999998e-05 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886524+0000", 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-05 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1673 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107627905000001, 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018154611000000001, 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1673 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868549+0000", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868550+0000", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868552+0000", 2024-09-21T22:50:30.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2589999999999999e-06 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868558+0000", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2040000000000002e-06 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868564+0000", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4489999999999996e-06 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886564+0000", 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017999549 2024-09-21T22:50:30.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886625+0000", 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1026999999999997e-05 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886704+0000", 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8805000000000002e-05 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1674 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107605275000001, 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018316941999999999, 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1674 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:30.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868572+0000", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3000000000000002e-07 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868574+0000", 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6929999999999999e-06 2024-09-21T22:50:30.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868579+0000", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1510000000000003e-06 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868583+0000", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8139999999999998e-06 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886746+0000", 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018163190999999999 2024-09-21T22:50:30.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886808+0000", 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1564999999999998e-05 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886889+0000", 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1198000000000004e-05 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1675 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107589704999999, 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018479580999999998, 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1675 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868587+0000", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868588+0000", 2024-09-21T22:50:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.91e-07 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868589+0000", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.576e-06 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868598+0000", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.551e-06 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868602+0000", 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.794e-06 2024-09-21T22:50:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886929+0000", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018327209000000001 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.886992+0000", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3553000000000006e-05 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887067+0000", 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4606999999999999e-05 2024-09-21T22:50:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1676 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107550643, 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018629687999999998, 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1676 2024-09-21T22:50:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868626+0000", 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868627+0000", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-07 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868629+0000", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1789999999999998e-06 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868636+0000", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868642+0000", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3579999999999999e-06 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887104+0000", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018462545 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887176+0000", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1540999999999998e-05 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887256+0000", 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0141000000000004e-05 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1694 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107467948, 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018819487999999999, 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1694 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868709+0000", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9700000000000003e-07 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868712+0000", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1619999999999998e-06 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868716+0000", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6099999999999999e-06 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868720+0000", 2024-09-21T22:50:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887298+0000", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018577750000000001 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887385+0000", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7077000000000007e-05 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887529+0000", 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014399200000000001 2024-09-21T22:50:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1695 3.1 3:87d5c231:::rbd_data.111ecbf88771.0000000000000002:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107449565, 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01924435, 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1695 2024-09-21T22:50:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868728+0000", 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6699999999999999e-07 2024-09-21T22:50:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868730+0000", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4830000000000002e-06 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868757+0000", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6149999999999999e-05 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868762+0000", 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9849999999999997e-06 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887608+0000", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018846471 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887781+0000", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017245300000000001 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.887972+0000", 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019134100000000001 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1699 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107407965, 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019655261, 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1699 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868769+0000", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868770+0000", 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0699999999999998e-07 2024-09-21T22:50:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868771+0000", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9300000000000002e-06 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868777+0000", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1270000000000002e-06 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868780+0000", 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6830000000000001e-06 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888128+0000", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019347907000000001 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888265+0000", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013713700000000001 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888424+0000", 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015907000000000001 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1700 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107392388000001, 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019917859999999999, 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1700 2024-09-21T22:50:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868785+0000", 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0199999999999998e-07 2024-09-21T22:50:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868786+0000", 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4750000000000001e-06 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868793+0000", 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868799+0000", 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.596e-06 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888513+0000", 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019714381999999999 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888606+0000", 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2514000000000002e-05 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888703+0000", 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6559999999999997e-05 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1701 3.1 3:8392de84:::rbd_data.111ecbf88771.0000000000000004:4 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107369943999998, 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020220140000000001, 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1701 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868807+0000", 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868808+0000", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868810+0000", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868816+0000", 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868820+0000", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7270000000000001e-06 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888804+0000", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019984927 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.888894+0000", 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9239000000000004e-05 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889027+0000", 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133566 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1704 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107352598999999, 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020436185999999999, 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1704 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868824+0000", 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868825+0000", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868826+0000", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.454e-06 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868831+0000", 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6009999999999997e-06 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868836+0000", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5230000000000003e-06 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889078+0000", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020241278000000001 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889176+0000", 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8041000000000005e-05 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889261+0000", 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4946999999999998e-05 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1705 3.1 3:86097379:::rbd_data.111ecbf88771.0000000000000006:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107267642, 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020613012999999999, 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1705 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868909+0000", 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868910+0000", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868914+0000", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9310000000000001e-06 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868925+0000", 2024-09-21T22:50:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0586e-05 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868933+0000", 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9689999999999994e-06 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889307+0000", 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020373901 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889399+0000", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2146999999999997e-05 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889522+0000", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012362500000000001 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1706 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107230521000002, 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021099340000000001, 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1706 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868947+0000", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868950+0000", 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9869999999999999e-06 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868959+0000", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9069999999999996e-06 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868965+0000", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2929999999999997e-06 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889619+0000", 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020653120000000001 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.889851+0000", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023190799999999999 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890046+0000", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000195389 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1707 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107202400999999, 2024-09-21T22:50:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022001837, 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1707 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868975+0000", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2500000000000005e-07 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868978+0000", 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8949999999999998e-06 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868989+0000", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1241e-05 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.868996+0000", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6499999999999999e-06 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890125+0000", 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021128457 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890278+0000", 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015349800000000001 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.890977+0000", 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069847099999999999 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1708 3.1 3:84e5a2c4:::rbd_data.111ecbf88771.0000000000000007:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.107170885999999, 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022287839, 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1708 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869006+0000", 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869007+0000", 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5000000000000003e-07 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869010+0000", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9469999999999999e-06 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869020+0000", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0316000000000001e-05 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.869027+0000", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5939999999999999e-06 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891011+0000", 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021984786999999999 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891138+0000", 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012628800000000001 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.891294+0000", 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015635700000000001 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1807 3.12 3:4e819a64:::rbd_data.111ea6ec99f1.000000000000009f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.911433732999999, 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027460622000000001, 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1807 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064743+0000", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064747+0000", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7929999999999999e-06 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064763+0000", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5603999999999998e-05 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064797+0000", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4319999999999997e-05 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.064833+0000", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5666e-05 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065001+0000", 2024-09-21T22:50:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016830799999999999 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065362+0000", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036063099999999998 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.070173+0000", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048112459999999999 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092106+0000", 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021932953000000002 2024-09-21T22:50:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092120+0000", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4318000000000001e-05 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092162+0000", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1980999999999997e-05 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.092204+0000", 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1801999999999997e-05 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3757 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.755615264999999, 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017800281000000001, 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3757 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220562+0000", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220563+0000", 2024-09-21T22:50:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220568+0000", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3299999999999998e-06 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220582+0000", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4047e-05 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220592+0000", 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5570000000000001e-06 2024-09-21T22:50:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230061+0000", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094694610000000002 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230174+0000", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011302300000000001 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230809+0000", 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063446900000000003 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232849+0000", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002040395 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238192+0000", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053432089999999998 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238279+0000", 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6624000000000003e-05 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238362+0000", 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3295999999999997e-05 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3758 3.1f 3:feaeeba6:::rbd_object_map.111e7572a6e.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.755563348000001, 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017995779, 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3758 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220614+0000", 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220615+0000", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2500000000000004e-07 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220620+0000", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5709999999999997e-06 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220636+0000", 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6030999999999999e-05 2024-09-21T22:50:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.220647+0000", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0411000000000001e-05 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231339+0000", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010691849999999999 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231406+0000", 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7846000000000001e-05 2024-09-21T22:50:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231663+0000", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025653299999999998 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.233751+0000", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020885869999999998 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238457+0000", 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047057230000000002 2024-09-21T22:50:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238543+0000", 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6191000000000002e-05 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238610+0000", 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6111000000000005e-05 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.063 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:31.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:31.065 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:31.061+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:31.165 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:31.161+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:31.241 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:31.241 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:31.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:31.305 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:31.301+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:31.367 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:31.361+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:31.379 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:31.380 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:31.380 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:31.380 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:31.380 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:31.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:31.465 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T22:50:31.468 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:31.461+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:31.468 INFO:teuthology.orchestra.run.smithi094.stdout:2029 write 0xb9a635 thru 0xba46ca (0xa096 bytes) 2024-09-21T22:50:31.484 INFO:teuthology.orchestra.run.smithi094.stdout:2032 write 0xad4ac31 thru 0xad59292 (0xe662 bytes) 2024-09-21T22:50:31.490 INFO:teuthology.orchestra.run.smithi094.stdout:2033 writesame 0x2195ec9 thru 0x21a5c7e data_size 0x1e(0xfdb6 bytes) 2024-09-21T22:50:31.536 INFO:teuthology.orchestra.run.smithi094.stdout:2035 read 0x54e347c thru 0x54ee45f (0xafe4 bytes) 2024-09-21T22:50:31.549 INFO:teuthology.orchestra.run.smithi094.stdout:2037 write 0xdf4ce3b thru 0xdf5c4f3 (0xf6b9 bytes) 2024-09-21T22:50:31.553 INFO:teuthology.orchestra.run.smithi094.stdout:2039 read 0x5153b1b thru 0x5162a08 (0xeeee bytes) 2024-09-21T22:50:31.554 INFO:teuthology.orchestra.run.smithi094.stdout:2040 trunc from 0xdf5c4f4 to 0xdbe5ebc 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:449 3.0 3:04e5c2a8:::rbd_data.111ecbf88771.0000000000000070:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.098663519999999, 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010416138, 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 449 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466071+0000", 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466074+0000", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7240000000000001e-06 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466093+0000", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9432999999999998e-05 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466112+0000", 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8777e-05 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466121+0000", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8699999999999998e-06 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466198+0000", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7168000000000004e-05 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466348+0000", 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150359 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.467428+0000", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010794089999999999 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476341+0000", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0089129889999999996 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476418+0000", 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6756999999999997e-05 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476450+0000", 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2313e-05 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476487+0000", 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7339000000000003e-05 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:450 3.0 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:31.572 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.098522829, 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010348049999999999, 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 450 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466212+0000", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6400000000000002e-07 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466217+0000", 2024-09-21T22:50:31.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1169999999999997e-06 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466230+0000", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3653e-05 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.466242+0000", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1479e-05 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472115+0000", 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058731429999999999 2024-09-21T22:50:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472225+0000", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110074 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.472818+0000", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059322200000000002 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476407+0000", 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035886849999999999 2024-09-21T22:50:31.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476497+0000", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0493000000000002e-05 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476536+0000", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8464999999999998e-05 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476560+0000", 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4054999999999999e-05 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:449 3.0 e28/24 ondisk, result = 0)", 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.095387984999999, 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070061350000000001, 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469347+0000", 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5899999999999995e-07 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469351+0000", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999998e-06 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469362+0000", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0709e-05 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.469369+0000", 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7839999999999998e-06 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476323+0000", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0069546770000000003 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476333+0000", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9620000000000005e-06 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.476353+0000", 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9544000000000001e-05 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:656 3.0 3:06e36901:::rbd_data.111ecbf88771.00000000000000fd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.842036403000002, 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080847060000000005, 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 656 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722698+0000", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722700+0000", 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9970000000000001e-06 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722709+0000", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8729999999999996e-06 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722736+0000", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6477000000000001e-05 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722748+0000", 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.722868+0000", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120562 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.723121+0000", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025297999999999998 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.724232+0000", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011103619999999999 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730644+0000", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064117250000000001 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730666+0000", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2220000000000001e-05 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730726+0000", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0235000000000002e-05 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.730783+0000", 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6885000000000002e-05 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1334 3.19 3:9fa18b7a:::rbd_data.111ea6ec99f1.00000000000001cc:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.774959749000001, 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085474309999999994, 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1334 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789775+0000", 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789777+0000", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1749999999999999e-06 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789787+0000", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9380000000000004e-06 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789810+0000", 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2864e-05 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789819+0000", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4739999999999994e-06 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.789889+0000", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9701000000000002e-05 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.790014+0000", 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000124595 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.790936+0000", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092237099999999995 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798087+0000", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071508090000000002 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798162+0000", 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5091000000000003e-05 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798255+0000", 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3528000000000003e-05 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.798322+0000", 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6884999999999994e-05 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1474 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.711463480999999, 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059933440000000003, 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1474 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:31.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853271+0000", 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853273+0000", 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9800000000000001e-06 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853280+0000", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.516e-06 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853300+0000", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0086999999999999e-05 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.853314+0000", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4426999999999999e-05 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.856768+0000", 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034537449999999998 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.856974+0000", 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020624200000000001 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859264+0000", 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022903469999999999 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1506 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.709546519, 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053295920000000002, 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1506 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.597 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855188+0000", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855189+0000", 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:50:31.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855195+0000", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855216+0000", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0336e-05 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.855230+0000", 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4163e-05 2024-09-21T22:50:31.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859943+0000", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047134539999999997 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860163+0000", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022026500000000001 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860518+0000", 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035424399999999998 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1602 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.705750130999999, 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051835220000000003, 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1602 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858984+0000", 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.858986+0000", 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1400000000000001e-06 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859005+0000", 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9714000000000001e-05 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859024+0000", 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8377e-05 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.859037+0000", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2977e-05 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.863598+0000", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045609120000000003 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.863829+0000", 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023124199999999999 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.864168+0000", 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00033916000000000003 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1633 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.704518001, 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052612559999999997, 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1633 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860217+0000", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860218+0000", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3430000000000001e-06 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860223+0000", 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9479999999999999e-06 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860241+0000", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7937000000000001e-05 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860254+0000", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3303e-05 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.864990+0000", 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047358749999999996 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865212+0000", 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022214500000000001 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865478+0000", 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026570499999999998 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1634 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.704462586999998, 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.006331957, 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1634 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860272+0000", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860273+0000", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860278+0000", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7090000000000001e-06 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860300+0000", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2110999999999999e-05 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860313+0000", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3117e-05 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865563+0000", 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052501570000000001 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.865781+0000", 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217378 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866604+0000", 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082327099999999999 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1665 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.704152984, 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063088179999999999, 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1665 2024-09-21T22:50:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860582+0000", 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860583+0000", 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3289999999999999e-06 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860588+0000", 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5400000000000003e-06 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860607+0000", 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8051e-05 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860620+0000", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3668e-05 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866187+0000", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055673019999999997 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866620+0000", 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043221299999999998 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.866890+0000", 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027071499999999999 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1666 3.19 3:9c07c7cc:::rbd_data.111ecbf88771.0000000000000003:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.704055881999999, 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0069166710000000001, 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1666 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860679+0000", 2024-09-21T22:50:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860680+0000", 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2550000000000001e-06 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860686+0000", 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4509999999999997e-06 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860707+0000", 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0937e-05 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.860722+0000", 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4717e-05 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867049+0000", 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063272010000000002 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867268+0000", 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021905899999999999 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.867595+0000", 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032705100000000002 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3483 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.370960316, 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014021865999999999, 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3483 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193774+0000", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193775+0000", 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0200000000000001e-07 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193779+0000", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3059999999999999e-06 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193788+0000", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0839999999999998e-06 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193796+0000", 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2320000000000004e-06 2024-09-21T22:50:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193947+0000", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015081400000000001 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194268+0000", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032190499999999999 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.207796+0000", 2024-09-21T22:50:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013527723 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3791 3.0 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.338752914000001, 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0068018389999999996, 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3791 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.225982+0000", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226369+0000", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00038704599999999999 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226383+0000", 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3188e-05 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226391+0000", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5960000000000004e-06 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226507+0000", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011558200000000001 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226671+0000", 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016398399999999999 2024-09-21T22:50:31.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227037+0000", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036616600000000002 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229514+0000", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002477431 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232671+0000", 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031565019999999998 2024-09-21T22:50:31.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232737+0000", 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6427000000000006e-05 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232784+0000", 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6143000000000002e-05 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3792 3.0 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.338316718, 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065155550000000001, 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3792 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226418+0000", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226419+0000", 2024-09-21T22:50:31.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1399999999999996e-07 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226431+0000", 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1836e-05 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226441+0000", 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0788000000000001e-05 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.226448+0000", 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5769999999999999e-06 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228354+0000", 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001906046 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228532+0000", 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000178479 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.228820+0000", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028807300000000002 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230390+0000", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001569254 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232844+0000", 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024540909999999998 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232892+0000", 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8229000000000002e-05 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.232933+0000", 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1368000000000001e-05 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3834 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.326164332999999, 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079824639999999999, 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3834 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238570+0000", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238573+0000", 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.266e-06 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238793+0000", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220805 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238816+0000", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3167e-05 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238834+0000", 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7050000000000001e-05 2024-09-21T22:50:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238939+0000", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105586 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.239061+0000", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012182500000000001 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.239332+0000", 2024-09-21T22:50:31.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000271418 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.244781+0000", 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054488289999999997 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246389+0000", 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001607863 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246484+0000", 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5174999999999996e-05 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.246553+0000", 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8479999999999995e-05 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3835 3.19 3:9c6f7810:::rbd_data.111e7572a6e.0000000000000004:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:31.644 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.325854352, 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010907765, 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3835 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:31.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238880+0000", 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238882+0000", 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-06 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238888+0000", 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2110000000000003e-06 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238904+0000", 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.238916+0000", 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1532000000000001e-05 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242095+0000", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031794459999999998 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242209+0000", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0001139 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.242442+0000", 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023316100000000001 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.245461+0000", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030189750000000001 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249562+0000", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041011789999999999 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249733+0000", 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017029499999999999 2024-09-21T22:50:31.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.249788+0000", 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5464999999999998e-05 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3860 3.19 3:9ca2baa7:::rbd_data.111e7572a6e.0000000000000011:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.323934878999999, 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010454101, 2024-09-21T22:50:31.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3860 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240800+0000", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240801+0000", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1170000000000001e-06 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240807+0000", 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.896e-06 2024-09-21T22:50:31.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240824+0000", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7226000000000001e-05 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.240858+0000", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3779999999999998e-05 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.244828+0000", 2024-09-21T22:50:31.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039701989999999998 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.245002+0000", 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017436 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.251254+0000", 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0062515230000000001 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:4610 3.19 3:9ad11ac6:::rbd_data.111e20a3ef21.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 378183] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.913333995, 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053903379999999997, 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 4610 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651401+0000", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651402+0000", 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-06 2024-09-21T22:50:31.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651603+0000", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020120000000000001 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651622+0000", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9389999999999999e-05 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651632+0000", 2024-09-21T22:50:31.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9599999999999995e-06 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.651770+0000", 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013722800000000001 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.652144+0000", 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037469100000000002 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-21T22:50:31.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.652867+0000", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072236399999999997 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.655650+0000", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027828990000000001 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656607+0000", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095755000000000005 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656708+0000", 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100572 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.656791+0000", 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3162000000000001e-05 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6028 3.0 3:074660d7:::rbd_data.111ebc806776.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:31.660 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.9277384509999997, 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012713250000000001, 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6028 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:31.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636996+0000", 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.636998+0000", 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.61e-06 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637005+0000", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8070000000000002e-06 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637018+0000", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3679999999999999e-05 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637026+0000", 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6020000000000002e-06 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637089+0000", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2994e-05 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.637144+0000", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5504999999999998e-05 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.639064+0000", 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019201649999999999 2024-09-21T22:50:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649635+0000", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010570296999999999 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649647+0000", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2262000000000001e-05 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649678+0000", 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1075000000000003e-05 2024-09-21T22:50:31.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:21.649709+0000", 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1253e-05 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:31.666 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2041 read 0x7254c2c thru 0x72550e5 (0x4ba bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2042 write 0xc7be071 thru 0xc7c643b (0x83cb bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2044 read 0x4cc31ff thru 0x4ccdfc7 (0xadc9 bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2045 read 0xabcc8a3 thru 0xabcf27c (0x29da bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2047 read 0xa2121b7 thru 0xa21d723 (0xb56d bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2049 read 0x842698 thru 0x844c96 (0x25ff bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2050 read 0x2c45235 thru 0x2c47f7b (0x2d47 bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2051 punch from 0x9364ed0 to 0x936c3b6, (0x74e6 bytes) 2024-09-21T22:50:31.668 INFO:teuthology.orchestra.run.smithi094.stdout:2053 write 0x7871e13 thru 0x7874b3e (0x2d2c bytes) 2024-09-21T22:50:31.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:31.670 INFO:teuthology.orchestra.run.smithi094.stdout:2054 read 0x3ef8110 thru 0x3ef85f7 (0x4e8 bytes) 2024-09-21T22:50:31.670 INFO:teuthology.orchestra.run.smithi094.stdout:2055 read 0x42809ff thru 0x4282620 (0x1c22 bytes) 2024-09-21T22:50:31.670 INFO:teuthology.orchestra.run.smithi094.stdout:2056 trunc from 0xdbe5ebc to 0x45e3c3c 2024-09-21T22:50:31.670 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:31.661+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:31.672 INFO:teuthology.orchestra.run.smithi094.stdout:2057 trunc from 0x45e3c3c to 0xd253cee 2024-09-21T22:50:31.685 INFO:teuthology.orchestra.run.smithi094.stdout:2058 punch from 0x372eb4c to 0x37305b9, (0x1a6d bytes) 2024-09-21T22:50:31.685 INFO:teuthology.orchestra.run.smithi094.stdout:2059 read 0x20213f7 thru 0x20255c0 (0x41ca bytes) 2024-09-21T22:50:31.686 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.686 INFO:teuthology.orchestra.run.smithi094.stdout:2064 read 0x3a3f002 thru 0x3a49e1e (0xae1d bytes) 2024-09-21T22:50:31.710 INFO:teuthology.orchestra.run.smithi094.stdout:2065 trunc from 0xd253cee to 0x3c4b826 2024-09-21T22:50:31.727 INFO:teuthology.orchestra.run.smithi094.stdout:2066 punch from 0x1e92640 to 0x1e9af99, (0x8959 bytes) 2024-09-21T22:50:31.727 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.727 INFO:teuthology.orchestra.run.smithi094.stdout:2068 read 0x1b526ab thru 0x1b5f01a (0xc970 bytes) 2024-09-21T22:50:31.727 INFO:teuthology.orchestra.run.smithi094.stdout:2069 read 0xbb8529 thru 0xbc7936 (0xf40e bytes) 2024-09-21T22:50:31.743 INFO:teuthology.orchestra.run.smithi094.stdout:2070 write 0x4395f52 thru 0x4399184 (0x3233 bytes) 2024-09-21T22:50:31.747 INFO:teuthology.orchestra.run.smithi094.stdout:2073 punch from 0x3d38eee to 0x3d4263f, (0x9751 bytes) 2024-09-21T22:50:31.769 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:31.765+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:31.773 INFO:teuthology.orchestra.run.smithi094.stdout:2074 writesame 0xaa809f8 thru 0xaa84144 data_size 0x63(0x374d bytes) 2024-09-21T22:50:31.773 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.792 INFO:teuthology.orchestra.run.smithi094.stdout:2076 write 0xe4fcc30 thru 0xe508fd3 (0xc3a4 bytes) 2024-09-21T22:50:31.800 INFO:teuthology.orchestra.run.smithi094.stdout:2077 read 0x55d6a64 thru 0x55d760b (0xba8 bytes) 2024-09-21T22:50:31.801 INFO:teuthology.orchestra.run.smithi094.stdout:2078 write 0x561fbb3 thru 0x5621b7f (0x1fcd bytes) 2024-09-21T22:50:31.805 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.805 INFO:teuthology.orchestra.run.smithi094.stdout:2081 read 0x27e0e0b thru 0x27eb7af (0xa9a5 bytes) 2024-09-21T22:50:31.806 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.806 INFO:teuthology.orchestra.run.smithi094.stdout:2083 write 0x3abf8de thru 0x3ac57be (0x5ee1 bytes) 2024-09-21T22:50:31.809 INFO:teuthology.orchestra.run.smithi094.stdout:2084 read 0x6eaad7d thru 0x6eb75f0 (0xc874 bytes) 2024-09-21T22:50:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:2087 writesame 0xcd5c394 thru 0xcd677b2 data_size 0x1(0xb41f bytes) 2024-09-21T22:50:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:31.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:31.859 INFO:teuthology.orchestra.run.smithi094.stdout:2088 read 0x40de8a8 thru 0x40edcef (0xf448 bytes) 2024-09-21T22:50:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:2091 punch from 0xc250895 to 0xc25ada2, (0xa50d bytes) 2024-09-21T22:50:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:2094 punch from 0xc7126aa to 0xc722672, (0xffc8 bytes) 2024-09-21T22:50:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:2095 read 0xb4ca786 thru 0xb4d20ba (0x7935 bytes) 2024-09-21T22:50:31.861 INFO:teuthology.orchestra.run.smithi094.stdout:2096 write 0x2f6e16e thru 0x2f74d6d (0x6c00 bytes) 2024-09-21T22:50:31.865 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:31.861+0000 7ffa211167c0 -1 osd.6 23 log_to_monitors true 2024-09-21T22:50:31.885 INFO:teuthology.orchestra.run.smithi094.stdout:2097 trunc from 0xe508fd4 to 0x4ebdb84 2024-09-21T22:50:31.885 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:31.881+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:31.901 INFO:teuthology.orchestra.run.smithi094.stdout:2098 read 0x1259b2c thru 0x1265aba (0xbf8f bytes) 2024-09-21T22:50:31.901 INFO:teuthology.orchestra.run.smithi094.stdout:2100 punch from 0xdf51c9 to 0xe003a7, (0xb1de bytes) 2024-09-21T22:50:31.902 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.902 INFO:teuthology.orchestra.run.smithi094.stdout:2102 read 0x3e934f8 thru 0x3e9fa03 (0xc50c bytes) 2024-09-21T22:50:31.902 INFO:teuthology.orchestra.run.smithi094.stdout:2104 writesame 0x27cc644 thru 0x27cea93 data_size 0x1c(0x2450 bytes) 2024-09-21T22:50:31.910 INFO:teuthology.orchestra.run.smithi094.stdout:2105 trunc from 0x4ebdb84 to 0x204c318 2024-09-21T22:50:31.921 INFO:teuthology.orchestra.run.smithi094.stdout:2106 punch from 0x812ffb to 0x81d792, (0xa797 bytes) 2024-09-21T22:50:31.922 INFO:teuthology.orchestra.run.smithi094.stdout:2107 read 0xddc074 thru 0xdeb5d8 (0xf565 bytes) 2024-09-21T22:50:31.922 INFO:teuthology.orchestra.run.smithi094.stdout:2110 read 0x1439e99 thru 0x143bc50 (0x1db8 bytes) 2024-09-21T22:50:31.923 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:31.923 INFO:teuthology.orchestra.run.smithi094.stdout:2114 read 0x853036 thru 0x862654 (0xf61f bytes) 2024-09-21T22:50:31.924 INFO:teuthology.orchestra.run.smithi094.stdout:2115 punch from 0x68a36a to 0x6902d9, (0x5f6f bytes) 2024-09-21T22:50:31.924 INFO:teuthology.orchestra.run.smithi094.stdout:2116 punch from 0x1af63ed to 0x1affb2d, (0x9740 bytes) 2024-09-21T22:50:31.924 INFO:teuthology.orchestra.run.smithi094.stdout:2117 clone 10 order 18 su 4096 sc 10 2024-09-21T22:50:31.970 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:31.965+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:31.986 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:31.986 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:31.986 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:31.986 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:31.986 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:32.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:32.072 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:32.069+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:32.172 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:50:32.173 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1020 3.0 e29/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 29'15, mlcod=28'12)", 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.003816860000001, 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018427630000000001, 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165434+0000", 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-21T22:50:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165447+0000", 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3157e-05 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165474+0000", 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6621000000000001e-05 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165487+0000", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3529999999999999e-05 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165644+0000", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015642 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165712+0000", 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8646000000000006e-05 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167238+0000", 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001525804 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167274+0000", 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6329999999999999e-05 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1030 3.0 e29/24 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head v 29'19, mlcod=29'13)", 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.999713878000001, 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022031799999999999, 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5499999999999999e-07 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169544+0000", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9320000000000002e-06 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169551+0000", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3209999999999999e-06 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169557+0000", 2024-09-21T22:50:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5779999999999998e-06 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169973+0000", 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00041665499999999999 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169994+0000", 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0679999999999999e-05 2024-09-21T22:50:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171635+0000", 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016405040000000001 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171738+0000", 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103155 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1349 3.0 e29/24 3:05bd433a:::rbd_data.111ea6ec99f1.00000000000000bb:head v 29'21, mlcod=29'19)", 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.335084632000001, 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020573869999999999, 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834165+0000", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6190000000000001e-06 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834177+0000", 2024-09-21T22:50:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.189e-05 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834200+0000", 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3110999999999999e-05 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834210+0000", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0009e-05 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834373+0000", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016245800000000001 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834430+0000", 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7402999999999999e-05 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836171+0000", 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017403659999999999 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836221+0000", 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0531000000000001e-05 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1848 3.4 e29/24 3:252aa742:::rbd_data.111ea6ec99f1.00000000000002cf:head v 29'14, mlcod=29'12)", 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.014025078, 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001883619, 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155226+0000", 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155264+0000", 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8418999999999999e-05 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155305+0000", 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1029999999999998e-05 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155329+0000", 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3623000000000001e-05 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155486+0000", 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015655899999999999 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155527+0000", 2024-09-21T22:50:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1083e-05 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157019+0000", 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001492555 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157107+0000", 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7723e-05 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1868 3.0 e29/24 3:031c5811:::rbd_data.111ea6ec99f1.00000000000001a4:head v 29'22, mlcod=29'20)", 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.985000952, 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016467299999999999, 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184247+0000", 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184250+0000", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6759999999999999e-06 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184311+0000", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0927000000000001e-05 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184344+0000", 2024-09-21T22:50:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3376e-05 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184360+0000", 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5109999999999999e-05 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184459+0000", 2024-09-21T22:50:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8964999999999999e-05 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.184487+0000", 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8092000000000001e-05 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185841+0000", 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013539789999999999 2024-09-21T22:50:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.185894+0000", 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3604999999999999e-05 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2008 3.4 e29/24 3:27f59992:::rbd_data.111ea6ec99f1.000000000000037d:head v 29'16, mlcod=29'14)", 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.775541027999999, 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016704179999999999, 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393709+0000", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0729999999999999e-06 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393727+0000", 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7227e-05 2024-09-21T22:50:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393753+0000", 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6066000000000001e-05 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393769+0000", 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.609e-05 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393939+0000", 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017003600000000001 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.394002+0000", 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3504999999999996e-05 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395320+0000", 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001317599 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395378+0000", 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7822000000000003e-05 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2114 3.9 e29/24 3:91282459:::rbd_data.111ea6ec99f1.00000000000002fb:head v 29'13, mlcod=29'11)", 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.606376782000002, 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001640273, 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562872+0000", 2024-09-21T22:50:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562874+0000", 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.03e-06 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562894+0000", 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0752999999999999e-05 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562914+0000", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9987e-05 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.562926+0000", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1153e-05 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563053+0000", 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127687 2024-09-21T22:50:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.563093+0000", 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9646e-05 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564463+0000", 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013701080000000001 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.564512+0000", 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8909000000000002e-05 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3136 3.9 e31/24 3:901d0113:::rbd_data.111ea6ec99f1.000000000000000d:head v 31'24, mlcod=30'21)", 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.974813936, 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021943800000000001, 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194440+0000", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5840000000000002e-06 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194452+0000", 2024-09-21T22:50:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1627999999999999e-05 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194459+0000", 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5119999999999997e-06 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194533+0000", 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4820000000000005e-05 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194575+0000", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1879999999999999e-05 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196580+0000", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020045559999999998 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196629+0000", 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9088000000000002e-05 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.213 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.942021395999999, 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:50:32.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3792 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'42, mlcod=32'38)", 2024-09-21T22:50:32.218 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.939356708, 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001678405, 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229894+0000", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9010000000000001e-06 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229904+0000", 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0808999999999999e-05 2024-09-21T22:50:32.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229926+0000", 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1778000000000001e-05 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229941+0000", 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4831000000000001e-05 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230115+0000", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017410799999999999 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230138+0000", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2714999999999999e-05 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231505+0000", 2024-09-21T22:50:32.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013669789999999999 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231570+0000", 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5283999999999994e-05 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3952 3.4 e32/24 3:27093f40:::rbd_data.111e7572a6e.0000000000000013:head v 32'39, mlcod=32'37)", 2024-09-21T22:50:32.223 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.910092924000001, 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019766520000000002, 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4e-07 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259228+0000", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1829999999999995e-05 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259244+0000", 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6028999999999998e-05 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259253+0000", 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5730000000000008e-06 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259328+0000", 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5209000000000004e-05 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259353+0000", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4626000000000001e-05 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261077+0000", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001724127 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261132+0000", 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5318000000000001e-05 2024-09-21T22:50:32.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4778 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'49, mlcod=32'47)", 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.152769402000001, 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024572740000000002, 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016481+0000", 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.813e-06 2024-09-21T22:50:32.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016608+0000", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012684600000000001 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016642+0000", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4150000000000003e-05 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016661+0000", 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8669999999999999e-05 2024-09-21T22:50:32.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016807+0000", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014669499999999999 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016875+0000", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8163000000000005e-05 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018819+0000", 2024-09-21T22:50:32.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019438350000000001 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018936+0000", 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117102 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4790 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'50, mlcod=32'48)", 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.135684691, 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001951752, 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033566+0000", 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.81e-06 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033609+0000", 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3550000000000001e-05 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033639+0000", 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9819999999999999e-05 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033653+0000", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3675999999999999e-05 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033764+0000", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011095300000000001 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033801+0000", 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7412e-05 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035439+0000", 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001638495 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035516+0000", 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6036000000000002e-05 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4796 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'51, mlcod=32'49)", 2024-09-21T22:50:32.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.115560453000001, 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001711647, 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:32.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053690+0000", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.244e-06 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053705+0000", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4856e-05 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053739+0000", 2024-09-21T22:50:32.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3543000000000001e-05 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053754+0000", 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5452999999999999e-05 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053901+0000", 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014656900000000001 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053967+0000", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6602999999999999e-05 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055327+0000", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013597329999999999 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055400+0000", 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2645999999999995e-05 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4809 3.9 e32/24 3:9625e1c1:::rbd_data.111e20a3ef21.00000000000000fb:head v 32'37, mlcod=32'35)", 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.096724475, 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016494910000000001, 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072524+0000", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072526+0000", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.86e-06 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072538+0000", 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1960000000000001e-05 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072565+0000", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.747e-05 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072584+0000", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8816000000000001e-05 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072721+0000", 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136823 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.072761+0000", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0386000000000003e-05 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.074090+0000", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013291900000000001 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.074173+0000", 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2985999999999995e-05 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4816 3.0 e32/24 3:048c4bde:::rbd_data.111e20a3ef21.0000000000000083:head v 32'69, mlcod=32'67)", 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.081098261999999, 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001935525, 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:32.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088152+0000", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-06 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088201+0000", 2024-09-21T22:50:32.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8479999999999997e-05 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088222+0000", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1137000000000001e-05 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088232+0000", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0267e-05 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088310+0000", 2024-09-21T22:50:32.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8208999999999996e-05 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088346+0000", 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5376000000000001e-05 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.089996+0000", 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016501300000000001 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.090086+0000", 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9975999999999994e-05 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:6552 3.0 e34/24 3:074660d7:::rbd_data.111ebc806776.0000000000000010:head v 34'96, mlcod=34'94)", 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1104704040000009, 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021954069999999999, 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058781+0000", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4739999999999999e-06 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058829+0000", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8155999999999998e-05 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058863+0000", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4139e-05 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058879+0000", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6214000000000001e-05 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059078+0000", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019923799999999999 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059120+0000", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1813000000000001e-05 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060834+0000", 2024-09-21T22:50:32.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017137140000000001 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060973+0000", 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013965900000000001 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:8685 3.9 e37/24 3:90954f07:::rbd_data.111e1fb363f5.0000000000000047:head v 37'46, mlcod=36'43)", 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.8354720110000002, 2024-09-21T22:50:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021535230000000001, 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:32.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333778+0000", 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.658e-06 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333789+0000", 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0791000000000001e-05 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333812+0000", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2782999999999999e-05 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333824+0000", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333967+0000", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014324199999999999 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.334030+0000", 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2139999999999998e-05 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335821+0000", 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001791579 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335930+0000", 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010884599999999999 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:9802 3.0 e38/24 3:05385c42:::rbd_data.111e33663d62.0000000000000056:head v 38'109, mlcod=38'107)", 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.6653671480000001, 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018689480000000001, 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503882+0000", 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7599999999999996e-07 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503889+0000", 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0169999999999999e-06 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503901+0000", 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1837e-05 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503909+0000", 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5909999999999998e-06 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504003+0000", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4301000000000004e-05 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504020+0000", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7472000000000001e-05 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.505708+0000", 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016874539999999999 2024-09-21T22:50:32.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.505750+0000", 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2400000000000001e-05 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:9799 3.9 e38/24 3:94cabc51:::rbd_data.111e33663d62.000000000000004c:head v 38'54, mlcod=38'50)", 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.6649431479999999, 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020855719999999999, 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504306+0000", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7000000000000003e-07 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504313+0000", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5849999999999996e-06 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504322+0000", 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.397e-06 2024-09-21T22:50:32.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504327+0000", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7720000000000002e-06 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504378+0000", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0846000000000001e-05 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504409+0000", 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0876000000000001e-05 2024-09-21T22:50:32.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.506344+0000", 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019355069999999999 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.506391+0000", 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6619000000000002e-05 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.266 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:32.267 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:32.169+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:32.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:32.268 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:32.201+0000 7ffa14ee3640 -1 osd.6 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:50:32.273 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:32.269+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:32.312 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone9 from 0x204c318 (overlap 0x204c318) to 0x102618c 2024-09-21T22:50:32.374 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:32.369+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:32.432 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T22:50:32.439 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:32.439 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:32.439 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:32.439 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:32.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:32.475 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:32.469+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:32.576 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:32.569+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:32.625 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:32.625 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:32.625 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:32.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:32.677 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:32.673+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:32.763 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:619 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.085690969000002, 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013060782, 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.764 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 619 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673457+0000", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.58e-07 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673459+0000", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6539999999999999e-06 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673465+0000", 2024-09-21T22:50:32.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.02e-06 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673470+0000", 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4479999999999996e-06 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682015+0000", 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085449849999999997 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682193+0000", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017775199999999999 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.683339+0000", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011463549999999999 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686337+0000", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00299797 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686383+0000", 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6462999999999999e-05 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686467+0000", 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3796999999999997e-05 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686517+0000", 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9879999999999997e-05 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:620 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.085671520999998, 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014257994, 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 620 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.770 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:32.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673479+0000", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5560000000000002e-06 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673495+0000", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.615e-05 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673500+0000", 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9439999999999996e-06 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684881+0000", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011381242 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684990+0000", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109197 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685289+0000", 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029849100000000001 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687532+0000", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022436510000000002 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687588+0000", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5655999999999997e-05 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687652+0000", 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3544e-05 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687734+0000", 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2126999999999995e-05 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:621 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.085640857000001, 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018613351, 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 621 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673507+0000", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673510+0000", 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8059999999999999e-06 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673516+0000", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3099999999999997e-06 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673520+0000", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4619999999999998e-06 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686563+0000", 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013042876 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686663+0000", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9570999999999999e-05 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687059+0000", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039618800000000001 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687883+0000", 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082348899999999995 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692061+0000", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004178022 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692094+0000", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3473999999999998e-05 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692120+0000", 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5711000000000002e-05 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.780 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:680 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.012964493999998, 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013778161000000001, 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 680 2024-09-21T22:50:32.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746187+0000", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9160000000000003e-06 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746193+0000", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746200+0000", 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5760000000000002e-06 2024-09-21T22:50:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753332+0000", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071319840000000001 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753409+0000", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7111000000000001e-05 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753698+0000", 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028885599999999999 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754451+0000", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075351400000000005 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759904+0000", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054529679999999999 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759937+0000", 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3018999999999997e-05 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759961+0000", 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3889000000000001e-05 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1046 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575844842999999, 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015794288, 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1046 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183304+0000", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1930000000000001e-06 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183311+0000", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3339999999999996e-06 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183339+0000", 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8030000000000001e-05 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183346+0000", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.192825+0000", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094795499999999998 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193017+0000", 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000192287 2024-09-21T22:50:32.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193797+0000", 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000779388 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197391+0000", 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035944969999999999 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198914+0000", 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015223039999999999 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199027+0000", 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113127 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199097+0000", 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0024999999999994e-05 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1047 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575792218, 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017348148000000001, 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1047 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.792 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183356+0000", 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:32.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183362+0000", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.806e-06 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183371+0000", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5529999999999998e-06 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183378+0000", 2024-09-21T22:50:32.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8689999999999998e-06 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194941+0000", 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011562767999999999 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195056+0000", 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115281 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195493+0000", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043665799999999998 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199120+0000", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036272919999999998 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200347+0000", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001226782 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200566+0000", 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021947500000000001 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200703+0000", 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013704200000000001 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1048 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575759893000001, 2024-09-21T22:50:32.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017543792999999998, 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1048 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183388+0000", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183392+0000", 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7139999999999999e-06 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183401+0000", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2469999999999997e-06 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183408+0000", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0469999999999996e-06 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195923+0000", 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012514658 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196021+0000", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8113000000000001e-05 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196394+0000", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037272199999999997 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200738+0000", 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043443179999999998 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200803+0000", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5166000000000006e-05 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200869+0000", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6190000000000002e-05 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200931+0000", 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1996999999999996e-05 2024-09-21T22:50:32.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1049 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575730258, 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017955736999999999, 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1049 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183418+0000", 2024-09-21T22:50:32.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183421+0000", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5580000000000001e-06 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183431+0000", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3430000000000002e-06 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183438+0000", 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-21T22:50:32.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197463+0000", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014025892999999999 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197596+0000", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013223600000000001 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198242+0000", 2024-09-21T22:50:32.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064663999999999998 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201081+0000", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002839144 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201120+0000", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8563000000000003e-05 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201262+0000", 2024-09-21T22:50:32.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014153000000000001 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201373+0000", 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111236 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1050 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575700397999999, 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020333185, 2024-09-21T22:50:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1050 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183448+0000", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183451+0000", 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6389999999999998e-06 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183460+0000", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183467+0000", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9600000000000003e-06 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199154+0000", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015686734000000001 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199259+0000", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105364 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199748+0000", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048820500000000002 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203483+0000", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003735787 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203519+0000", 2024-09-21T22:50:32.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5954000000000002e-05 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203653+0000", 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013328799999999999 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203780+0000", 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012739399999999999 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.813 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1051 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575647016000001, 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021843896000000002, 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1051 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183501+0000", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2399999999999997e-07 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183506+0000", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183527+0000", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1353e-05 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183535+0000", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0180000000000001e-06 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201414+0000", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01787888 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201496+0000", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2201999999999997e-05 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201824+0000", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032739100000000001 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205098+0000", 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032737280000000001 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205130+0000", 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2805000000000002e-05 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205258+0000", 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127911 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205344+0000", 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5882000000000002e-05 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1052 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575599189999998, 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022039640999999999, 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1052 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183549+0000", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183553+0000", 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1130000000000003e-06 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183563+0000", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6930000000000002e-06 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183570+0000", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202302+0000", 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018731863000000001 2024-09-21T22:50:32.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202413+0000", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111117 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202922+0000", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050897300000000002 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205382+0000", 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024604240000000001 2024-09-21T22:50:32.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205403+0000", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1058999999999999e-05 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205503+0000", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100068 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205588+0000", 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4584000000000002e-05 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1053 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.575568100000002, 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022880587000000001, 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1053 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183580+0000", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183592+0000", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.224e-05 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183602+0000", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094e-05 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183609+0000", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1810000000000004e-06 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203845+0000", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020235907000000001 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203896+0000", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1187999999999998e-05 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204112+0000", 2024-09-21T22:50:32.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021558999999999999 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205609+0000", 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001496743 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206370+0000", 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076130800000000004 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206421+0000", 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0491000000000002e-05 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206460+0000", 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9218999999999998e-05 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1054 3.2 3:46641ad1:::rbd_data.111ecbf88771.0000000000000000:head [stat,truncate 204800] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.571371288999998, 2024-09-21T22:50:32.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019736885999999999, 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1054 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187777+0000", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187783+0000", 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2780000000000001e-06 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187794+0000", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1464999999999999e-05 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187811+0000", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6716999999999999e-05 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204451+0000", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016639856000000001 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204504+0000", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3763e-05 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204604+0000", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9433999999999997e-05 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205621+0000", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010175049999999999 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207340+0000", 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718884 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207409+0000", 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8959000000000002e-05 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207513+0000", 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010366600000000001 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1805 3.b 3:d5e4ffc8:::rbd_data.111ea6ec99f1.000000000000009d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.700406102999999, 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.032681931999999997, 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1805 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058743+0000", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.37e-06 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058751+0000", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058769+0000", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7927e-05 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058780+0000", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1353e-05 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058934+0000", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015403899999999999 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.059298+0000", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036453700000000001 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065146+0000", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058479789999999997 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091319+0000", 2024-09-21T22:50:32.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026172104000000002 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091334+0000", 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4939999999999999e-05 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091379+0000", 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5593000000000002e-05 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091423+0000", 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4014000000000002e-05 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3479 3.7 3:e3964c56:::rbd_data.111e7572a6e.0000000000000000:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.565524904, 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01465428, 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3479 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193624+0000", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9199999999999998e-06 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193634+0000", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0132999999999999e-05 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193666+0000", 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1405999999999997e-05 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193684+0000", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8185999999999998e-05 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193764+0000", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0051999999999999e-05 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193901+0000", 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013736 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.208277+0000", 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014375222999999999 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3487 3.b 3:d69a8a6f:::rbd_data.111e7572a6e.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.564584248999999, 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01338204, 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:32.845 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3487 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194564+0000", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4470000000000001e-06 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194573+0000", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-06 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194596+0000", 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3312999999999999e-05 2024-09-21T22:50:32.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194611+0000", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194681+0000", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9865999999999998e-05 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194803+0000", 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012176900000000001 2024-09-21T22:50:32.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.207945+0000", 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013141972 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6462 3.2 3:44ccee00:::rbd_data.111ebc806776.000000000000001a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:32.849 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.9969751139999996, 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.08074365, 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6462 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762174+0000", 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3029999999999998e-06 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762186+0000", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.136e-05 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762217+0000", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0663000000000001e-05 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762232+0000", 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5099999999999999e-05 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762388+0000", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000156362 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762632+0000", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243995 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.777316+0000", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014684262999999999 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842690+0000", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065373938000000006 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842722+0000", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2283999999999998e-05 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842825+0000", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102192 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842916+0000", 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1189999999999999e-05 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:6462 3.2 e34/24 ondisk, result = 0)", 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.9375612120000003, 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021121876000000001, 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821588+0000", 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0600000000000002e-06 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821598+0000", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8160000000000005e-06 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821633+0000", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4975999999999999e-05 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821651+0000", 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8287e-05 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842659+0000", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021007426999999999 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842677+0000", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8533e-05 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842708+0000", 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0777e-05 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:9376 3.2 3:4315e470:::rbd_data.111e33663d62.0000000000000004:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.3422520909999998, 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013487305, 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 9376 2024-09-21T22:50:32.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416896+0000", 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8500000000000001e-07 2024-09-21T22:50:32.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416900+0000", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7450000000000001e-06 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416907+0000", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7360000000000001e-06 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416913+0000", 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.418988+0000", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020751480000000002 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419086+0000", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7659999999999997e-05 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419282+0000", 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019622899999999999 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.420737+0000", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455582 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430236+0000", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094984900000000001 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430316+0000", 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0382000000000004e-05 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430383+0000", 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6178000000000002e-05 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:11367 3.1e 3:7cc77dab:::rbd_data.111ea12b32db.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.94467778099999999, 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044095079000000002, 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:32.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11367 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:32.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814472+0000", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0930000000000001e-06 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814483+0000", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1293e-05 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814512+0000", 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9373000000000002e-05 2024-09-21T22:50:32.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814527+0000", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4589999999999999e-05 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814641+0000", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114339 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814787+0000", 2024-09-21T22:50:32.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014541900000000001 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858463+0000", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043676404000000002 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858508+0000", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4731000000000003e-05 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858565+0000", 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6836999999999998e-05 2024-09-21T22:50:32.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.869 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:32.869 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:32.869 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:32.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2118 writesame 0xb221844 thru 0xb225ef5 data_size 0x1(0x46b2 bytes) 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2119 read 0xa1a935f thru 0xa1b8d79 (0xfa1b bytes) 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2120 write 0xa61efec thru 0xa62c936 (0xd94b bytes) 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2121 punch from 0x7a19576 to 0x7a28e26, (0xf8b0 bytes) 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2122 read 0x3d0538d thru 0x3d10ac2 (0xb736 bytes) 2024-09-21T22:50:32.871 INFO:teuthology.orchestra.run.smithi094.stdout:2123 write 0x41f5142 thru 0x4204500 (0xf3bf bytes) 2024-09-21T22:50:32.871 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:32.865+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:32.890 INFO:teuthology.orchestra.run.smithi094.stdout:2125 write 0x8b682fa thru 0x8b6a884 (0x258b bytes) 2024-09-21T22:50:32.898 INFO:teuthology.orchestra.run.smithi094.stdout:2126 writesame 0x8b6f6fa thru 0x8b76ed9 data_size 0x38(0x77e0 bytes) 2024-09-21T22:50:32.908 INFO:teuthology.orchestra.run.smithi094.stdout:2128 read 0x8e694ea thru 0x8e78f2a (0xfa41 bytes) 2024-09-21T22:50:32.908 INFO:teuthology.orchestra.run.smithi094.stdout:2129 write 0x4a5d904 thru 0x4a61b0f (0x420c bytes) 2024-09-21T22:50:32.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:32.917 INFO:teuthology.orchestra.run.smithi094.stdout:2131 write 0x8af01f6 thru 0x8afaf2a (0xad35 bytes) 2024-09-21T22:50:32.927 INFO:teuthology.orchestra.run.smithi094.stdout:2132 write 0x42bc5ee thru 0x42cabb1 (0xe5c4 bytes) 2024-09-21T22:50:32.934 INFO:teuthology.orchestra.run.smithi094.stdout:2133 read 0x7b1387d thru 0x7b1b252 (0x79d6 bytes) 2024-09-21T22:50:32.948 INFO:teuthology.orchestra.run.smithi094.stdout:2135 trunc from 0xb225ef6 to 0x293562a 2024-09-21T22:50:32.970 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:32.965+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:32.986 INFO:teuthology.orchestra.run.smithi094.stdout:2136 write 0x2942861 thru 0x2944fcc (0x276c bytes) 2024-09-21T22:50:32.992 INFO:teuthology.orchestra.run.smithi094.stdout:2137 read 0x174df32 thru 0x175d209 (0xf2d8 bytes) 2024-09-21T22:50:32.998 INFO:teuthology.orchestra.run.smithi094.stdout:2138 punch from 0x2011f07 to 0x201c0fb, (0xa1f4 bytes) 2024-09-21T22:50:33.005 INFO:teuthology.orchestra.run.smithi094.stdout:2139 trunc from 0x2944fcd to 0x7f865fc 2024-09-21T22:50:33.013 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:33.013 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:33.013 INFO:teuthology.orchestra.run.smithi094.stdout:2140 writesame 0x2666171 thru 0x266f398 data_size 0xc(0x9228 bytes) 2024-09-21T22:50:33.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:33.032 INFO:teuthology.orchestra.run.smithi094.stdout:2141 read 0x36ca5db thru 0x36d6596 (0xbfbc bytes) 2024-09-21T22:50:33.042 INFO:teuthology.orchestra.run.smithi094.stdout:2142 trunc from 0x7f865fc to 0xa26dce 2024-09-21T22:50:33.079 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:33.081 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:33.077+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:33.096 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:33.114 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:33.115 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:33.131 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:33.132 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:33.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:33.168 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:50:33.170 INFO:teuthology.orchestra.run.smithi094.stdout:2143 trunc from 0xa26dce to 0x7e35674 2024-09-21T22:50:33.172 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:33.165+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:33.174 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:33.183 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.183 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:33.187 INFO:teuthology.orchestra.run.smithi094.stdout:2145 write 0x1dced41 thru 0x1dcf252 (0x512 bytes) 2024-09-21T22:50:33.200 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-21T22:50:33.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:33.203 INFO:teuthology.orchestra.run.smithi094.stdout:2148 write 0xbf34ed6 thru 0xbf39790 (0x48bb bytes) 2024-09-21T22:50:33.211 INFO:teuthology.orchestra.run.smithi094.stdout:2149 write 0xa6b21f3 thru 0xa6b667d (0x448b bytes) 2024-09-21T22:50:33.218 INFO:teuthology.orchestra.run.smithi094.stdout:2150 read 0x226f463 thru 0x2278488 (0x9026 bytes) 2024-09-21T22:50:33.218 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:33.218 INFO:teuthology.orchestra.run.smithi094.stdout:2152 writesame 0xb8ff1cf thru 0xb906197 data_size 0x1(0x6fc9 bytes) 2024-09-21T22:50:33.221 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:33.222 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:33.284 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:33.281+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:33.358 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.358 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:33.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:33.374 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:33.369+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:33.475 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:33.469+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:33.495 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.496 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:33.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:33.576 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:33.569+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:33.672 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.672 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:33.676 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:33.669+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:33.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:33.778 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:33.777+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:33.854 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.855 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:33.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:33.878 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:33.873+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:33.979 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:33.973+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:33.988 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:33.988 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:34.080 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:34.073+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:34.126 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.126 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:34.181 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:34.177+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:34.195 INFO:teuthology.orchestra.run.smithi094.stdout:2153 write 0xa3b05e2 thru 0xa3b6373 (0x5d92 bytes) 2024-09-21T22:50:34.283 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:34.277+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:34.304 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.304 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:34.384 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:34.377+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:34.449 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.449 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:34.513 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:34.509+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:34.585 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:34.581+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:34.586 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.586 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:34.686 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:34.681+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:34.760 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.760 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:34.817 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:34.813+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:34.887 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:34.881+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:34.941 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:34.941 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:34.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:35.001 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:34.997+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:35.089 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:35.085+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:35.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:35.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:50:35.217 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:35.213+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:35.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:35.291 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:35.291 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:35.291 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:35.285+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:35.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:35.349 INFO:teuthology.orchestra.run.smithi094.stdout:2155 write 0xdde5da1 thru 0xdde7d99 (0x1ff9 bytes) 2024-09-21T22:50:35.364 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.364 INFO:teuthology.orchestra.run.smithi094.stdout:2157 read 0xa66a57f thru 0xa6789a4 (0xe426 bytes) 2024-09-21T22:50:35.365 INFO:teuthology.orchestra.run.smithi094.stdout:2158 write 0x44b0d0a thru 0x44b7106 (0x63fd bytes) 2024-09-21T22:50:35.392 INFO:teuthology.orchestra.run.smithi094.stdout:2159 read 0x6fe82 thru 0x70332 (0x4b1 bytes) 2024-09-21T22:50:35.392 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:35.389+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:35.394 INFO:teuthology.orchestra.run.smithi094.stdout:2160 read 0xbe48bb9 thru 0xbe57318 (0xe760 bytes) 2024-09-21T22:50:35.395 INFO:teuthology.orchestra.run.smithi094.stdout:2161 read 0xd5f9db5 thru 0xd5ffbb4 (0x5e00 bytes) 2024-09-21T22:50:35.396 INFO:teuthology.orchestra.run.smithi094.stdout:2164 read 0xcb2d549 thru 0xcb34600 (0x70b8 bytes) 2024-09-21T22:50:35.396 INFO:teuthology.orchestra.run.smithi094.stdout:2165 writesame 0xa219f3f thru 0xa21d519 data_size 0x1(0x35db bytes) 2024-09-21T22:50:35.411 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:35.411 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:2166 trunc from 0xdde7d9a to 0xd9bc851 2024-09-21T22:50:35.431 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:2167 writesame 0xbb06377 thru 0xbb0c3b0 data_size 0x71(0x603a bytes) 2024-09-21T22:50:35.451 INFO:teuthology.orchestra.run.smithi094.stdout:2168 punch from 0x70641e to 0x70d0ac, (0x6c8e bytes) 2024-09-21T22:50:35.451 INFO:teuthology.orchestra.run.smithi094.stdout:2169 write 0xc86b7dc thru 0xc872bc8 (0x73ed bytes) 2024-09-21T22:50:35.451 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:35.473 INFO:teuthology.orchestra.run.smithi094.stdout:2170 read 0x9855d42 thru 0x9864501 (0xe7c0 bytes) 2024-09-21T22:50:35.473 INFO:teuthology.orchestra.run.smithi094.stdout:2171 read 0x6afc2fd thru 0x6b0293d (0x6641 bytes) 2024-09-21T22:50:35.473 INFO:teuthology.orchestra.run.smithi094.stdout:2172 read 0x47ce947 thru 0x47d2cc3 (0x437d bytes) 2024-09-21T22:50:35.473 INFO:teuthology.orchestra.run.smithi094.stdout:2173 read 0x7b47e7a thru 0x7b53ecc (0xc053 bytes) 2024-09-21T22:50:35.474 INFO:teuthology.orchestra.run.smithi094.stdout:2174 trunc from 0xd9bc851 to 0xcc22dfe 2024-09-21T22:50:35.484 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:35.485 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:35.485 INFO:teuthology.orchestra.run.smithi094.stdout:2176 read 0x7e5d3e thru 0x7ecce8 (0x6fab bytes) 2024-09-21T22:50:35.487 INFO:teuthology.orchestra.run.smithi094.stdout:2177 write 0x8aebbef thru 0x8af3608 (0x7a1a bytes) 2024-09-21T22:50:35.493 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:35.489+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:35.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:35.513 INFO:teuthology.orchestra.run.smithi094.stdout:2179 punch from 0x51fa986 to 0x51fcaf8, (0x2172 bytes) 2024-09-21T22:50:35.523 INFO:teuthology.orchestra.run.smithi094.stdout:2180 write 0xe3dbde9 thru 0xe3ea010 (0xe228 bytes) 2024-09-21T22:50:35.548 INFO:teuthology.orchestra.run.smithi094.stdout:2182 writesame 0x8cc1aa thru 0x8d044b data_size 0x3(0x42a2 bytes) 2024-09-21T22:50:35.595 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:35.593+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:35.626 INFO:teuthology.orchestra.run.smithi094.stdout:2183 write 0x2feaf80 thru 0x2ff19cd (0x6a4e bytes) 2024-09-21T22:50:35.633 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:35.633 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:35.641 INFO:teuthology.orchestra.run.smithi094.stdout:2184 writesame 0xc41c946 thru 0xc423a36 data_size 0x1(0x70f1 bytes) 2024-09-21T22:50:35.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:35.659 INFO:teuthology.orchestra.run.smithi094.stdout:2185 read 0x4222f50 thru 0x42327f5 (0xf8a6 bytes) 2024-09-21T22:50:35.659 INFO:teuthology.orchestra.run.smithi094.stdout:2186 write 0x1c28121 thru 0x1c2e75d (0x663d bytes) 2024-09-21T22:50:35.671 INFO:teuthology.orchestra.run.smithi094.stdout:2188 writesame 0x5e9809b thru 0x5e9a0bb data_size 0x1(0x2021 bytes) 2024-09-21T22:50:35.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:35.674 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:35.679 INFO:teuthology.orchestra.run.smithi094.stdout:2190 write 0x6869ab3 thru 0x686cc19 (0x3167 bytes) 2024-09-21T22:50:35.688 INFO:teuthology.orchestra.run.smithi094.stdout:2191 punch from 0x13f9bcc to 0x13fe6cc, (0x4b00 bytes) 2024-09-21T22:50:35.688 INFO:teuthology.orchestra.run.smithi094.stdout:2192 write 0xceebda4 thru 0xceee74f (0x29ac bytes) 2024-09-21T22:50:35.695 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:35.689+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:35.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:35.696 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:35.699 INFO:teuthology.orchestra.run.smithi094.stdout:2194 read 0x8c02483 thru 0x8c09312 (0x6e90 bytes) 2024-09-21T22:50:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.700 INFO:teuthology.orchestra.run.smithi094.stdout:2199 writesame 0x82b1241 thru 0x82baeea data_size 0xb(0x9caa bytes) 2024-09-21T22:50:35.715 INFO:teuthology.orchestra.run.smithi094.stdout:2201 writesame 0xbdf2960 thru 0xbdf5a38 data_size 0x29(0x30d9 bytes) 2024-09-21T22:50:35.723 INFO:teuthology.orchestra.run.smithi094.stdout:2203 writesame 0x933bf91 thru 0x933fdb6 data_size 0x2b(0x3e26 bytes) 2024-09-21T22:50:35.730 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:35.731 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:35.735 INFO:teuthology.orchestra.run.smithi094.stdout:2204 read 0x152f8ad thru 0x153ab3f (0xb293 bytes) 2024-09-21T22:50:35.735 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.735 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.735 INFO:teuthology.orchestra.run.smithi094.stdout:2209 writesame 0x834f54f thru 0x835e85c data_size 0x2(0xf30e bytes) 2024-09-21T22:50:35.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:35.753 INFO:teuthology.orchestra.run.smithi094.stdout:2210 read 0x4e0e5ad thru 0x4e18f68 (0xa9bc bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2211 read 0x6f8999e thru 0x6f9506c (0xb6cf bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2215 punch from 0x43f8408 to 0x43fa4e8, (0x20e0 bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2220 read 0x6048240 thru 0x605185f (0x9620 bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2221 read 0x8c25465 thru 0x8c28b48 (0x36e4 bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2222 punch from 0xc4a2a0 to 0xc56ee2, (0xcc42 bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2223 punch from 0x256222d to 0x256d43b, (0xb20e bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2225 read 0x759b0c0 thru 0x759fdc2 (0x4d03 bytes) 2024-09-21T22:50:35.754 INFO:teuthology.orchestra.run.smithi094.stdout:2226 trunc from 0xe3ea011 to 0xdcfc2a9 2024-09-21T22:50:35.758 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:35.758 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:35.775 INFO:teuthology.orchestra.run.smithi094.stdout:2227 read 0x5949481 thru 0x59542dc (0xae5c bytes) 2024-09-21T22:50:35.775 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:35.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:35.776 INFO:teuthology.orchestra.run.smithi094.stdout:2228 write 0x8361bb1 thru 0x836ad88 (0x91d8 bytes) 2024-09-21T22:50:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:2229 write 0xb150b27 thru 0xb152afa (0x1fd4 bytes) 2024-09-21T22:50:35.791 INFO:teuthology.orchestra.run.smithi094.stdout:2230 punch from 0x42838c3 to 0x428ae7b, (0x75b8 bytes) 2024-09-21T22:50:35.791 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.791 INFO:teuthology.orchestra.run.smithi094.stdout:2232 punch from 0x6c756e2 to 0x6c7edda, (0x96f8 bytes) 2024-09-21T22:50:35.791 INFO:teuthology.orchestra.run.smithi094.stdout:2234 read 0x4f353ac thru 0x4f3c751 (0x73a6 bytes) 2024-09-21T22:50:35.791 INFO:teuthology.orchestra.run.smithi094.stdout:2236 read 0xc8d2e7d thru 0xc8e16e8 (0xe86c bytes) 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1020 3.0 e29/24 3:0160ec32:::rbd_data.111ecbf88771.0000000000000045:head v 29'15, mlcod=28'12)", 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.628589719000001, 2024-09-21T22:50:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018427630000000001, 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165432+0000", 2024-09-21T22:50:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165434+0000", 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165447+0000", 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3157e-05 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165474+0000", 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6621000000000001e-05 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165487+0000", 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3529999999999999e-05 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165644+0000", 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015642 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.165712+0000", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8646000000000006e-05 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167238+0000", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001525804 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.167274+0000", 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6329999999999999e-05 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1030 3.0 e29/24 3:027fef3a:::rbd_data.111ecbf88771.0000000000000071:head v 29'19, mlcod=29'13)", 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.624486737000002, 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022031799999999999, 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169535+0000", 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5499999999999999e-07 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169544+0000", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9320000000000002e-06 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169551+0000", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3209999999999999e-06 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169557+0000", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5779999999999998e-06 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169973+0000", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00041665499999999999 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.169994+0000", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0679999999999999e-05 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171635+0000", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016405040000000001 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.171738+0000", 2024-09-21T22:50:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103155 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1349 3.0 e29/24 3:05bd433a:::rbd_data.111ea6ec99f1.00000000000000bb:head v 29'21, mlcod=29'19)", 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.959857491000001, 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020573869999999999, 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834164+0000", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834165+0000", 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6190000000000001e-06 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834177+0000", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.189e-05 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834200+0000", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3110999999999999e-05 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834210+0000", 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0009e-05 2024-09-21T22:50:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834373+0000", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016245800000000001 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.834430+0000", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7402999999999999e-05 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836171+0000", 2024-09-21T22:50:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017403659999999999 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.836221+0000", 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0531000000000001e-05 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:1848 3.4 e29/24 3:252aa742:::rbd_data.111ea6ec99f1.00000000000002cf:head v 29'14, mlcod=29'12)", 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.638797937, 2024-09-21T22:50:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001883619, 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155223+0000", 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155226+0000", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155264+0000", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8418999999999999e-05 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155305+0000", 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1029999999999998e-05 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155329+0000", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3623000000000001e-05 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155486+0000", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015655899999999999 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.155527+0000", 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1083e-05 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157019+0000", 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001492555 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.157107+0000", 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7723e-05 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:2008 3.4 e29/24 3:27f59992:::rbd_data.111ea6ec99f1.000000000000037d:head v 29'16, mlcod=29'14)", 2024-09-21T22:50:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.400313886999999, 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016704179999999999, 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393707+0000", 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393709+0000", 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0729999999999999e-06 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393727+0000", 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7227e-05 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393753+0000", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6066000000000001e-05 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393769+0000", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.609e-05 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.393939+0000", 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017003600000000001 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.394002+0000", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3504999999999996e-05 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395320+0000", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001317599 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.395378+0000", 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7822000000000003e-05 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3136 3.9 e31/24 3:901d0113:::rbd_data.111ea6ec99f1.000000000000000d:head v 31'24, mlcod=30'21)", 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.599586795, 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021943800000000001, 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194435+0000", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194440+0000", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5840000000000002e-06 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194452+0000", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1627999999999999e-05 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194459+0000", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5119999999999997e-06 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194533+0000", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4820000000000005e-05 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.194575+0000", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1879999999999999e-05 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196580+0000", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020045559999999998 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:17.196629+0000", 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9088000000000002e-05 2024-09-21T22:50:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.566794255000001, 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:50:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3792 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'42, mlcod=32'38)", 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.564129566999998, 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001678405, 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229892+0000", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229894+0000", 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9010000000000001e-06 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229904+0000", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0808999999999999e-05 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229926+0000", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1778000000000001e-05 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.229941+0000", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4831000000000001e-05 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230115+0000", 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017410799999999999 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230138+0000", 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2714999999999999e-05 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231505+0000", 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013669789999999999 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231570+0000", 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5283999999999994e-05 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3952 3.4 e32/24 3:27093f40:::rbd_data.111e7572a6e.0000000000000013:head v 32'39, mlcod=32'37)", 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.534865783000001, 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019766520000000002, 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259156+0000", 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4e-07 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259228+0000", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1829999999999995e-05 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259244+0000", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6028999999999998e-05 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259253+0000", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5730000000000008e-06 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259328+0000", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5209000000000004e-05 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.259353+0000", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4626000000000001e-05 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261077+0000", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001724127 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.261132+0000", 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5318000000000001e-05 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4778 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'49, mlcod=32'47)", 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.777542261000001, 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024572740000000002, 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016479+0000", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016481+0000", 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.813e-06 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016608+0000", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012684600000000001 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016642+0000", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4150000000000003e-05 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016661+0000", 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8669999999999999e-05 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016807+0000", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014669499999999999 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.016875+0000", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8163000000000005e-05 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018819+0000", 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019438350000000001 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.018936+0000", 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117102 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4790 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'50, mlcod=32'48)", 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.760457550000002, 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001951752, 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033564+0000", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033566+0000", 2024-09-21T22:50:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.81e-06 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033609+0000", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3550000000000001e-05 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033639+0000", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9819999999999999e-05 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033653+0000", 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3675999999999999e-05 2024-09-21T22:50:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033764+0000", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011095300000000001 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.033801+0000", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7412e-05 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035439+0000", 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001638495 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.035516+0000", 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6036000000000002e-05 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4796 3.4 e32/24 3:239fe3a8:::rbd_data.111e20a3ef21.00000000000001dc:head v 32'51, mlcod=32'49)", 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.740333312000001, 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001711647, 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053688+0000", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053690+0000", 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.244e-06 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053705+0000", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4856e-05 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053739+0000", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3543000000000001e-05 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053754+0000", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5452999999999999e-05 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053901+0000", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014656900000000001 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.053967+0000", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6602999999999999e-05 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055327+0000", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013597329999999999 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.055400+0000", 2024-09-21T22:50:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2645999999999995e-05 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:4816 3.0 e32/24 3:048c4bde:::rbd_data.111e20a3ef21.0000000000000083:head v 32'69, mlcod=32'67)", 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.705871121000001, 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001935525, 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088150+0000", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088152+0000", 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-06 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088201+0000", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8479999999999997e-05 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088222+0000", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1137000000000001e-05 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088232+0000", 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0267e-05 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088310+0000", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8208999999999996e-05 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.088346+0000", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5376000000000001e-05 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.089996+0000", 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016501300000000001 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:19.090086+0000", 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9975999999999994e-05 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:6552 3.0 e34/24 3:074660d7:::rbd_data.111ebc806776.0000000000000010:head v 34'96, mlcod=34'94)", 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.735243262999999, 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021954069999999999, 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058778+0000", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058781+0000", 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4739999999999999e-06 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058829+0000", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8155999999999998e-05 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058863+0000", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4139e-05 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.058879+0000", 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6214000000000001e-05 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059078+0000", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019923799999999999 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.059120+0000", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1813000000000001e-05 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060834+0000", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017137140000000001 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:23.060973+0000", 2024-09-21T22:50:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013965900000000001 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:8685 3.9 e37/24 3:90954f07:::rbd_data.111e1fb363f5.0000000000000047:head v 37'46, mlcod=36'43)", 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.4602448700000004, 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021535230000000001, 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333776+0000", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333778+0000", 2024-09-21T22:50:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.658e-06 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333789+0000", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0791000000000001e-05 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333812+0000", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2782999999999999e-05 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333824+0000", 2024-09-21T22:50:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.333967+0000", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014324199999999999 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.334030+0000", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2139999999999998e-05 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335821+0000", 2024-09-21T22:50:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001791579 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:27.335930+0000", 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010884599999999999 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:9802 3.0 e38/24 3:05385c42:::rbd_data.111e33663d62.0000000000000056:head v 38'109, mlcod=38'107)", 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.2901400069999998, 2024-09-21T22:50:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018689480000000001, 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503881+0000", 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503882+0000", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7599999999999996e-07 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503889+0000", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0169999999999999e-06 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503901+0000", 2024-09-21T22:50:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1837e-05 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.503909+0000", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5909999999999998e-06 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504003+0000", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4301000000000004e-05 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504020+0000", 2024-09-21T22:50:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7472000000000001e-05 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.505708+0000", 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016874539999999999 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.505750+0000", 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2400000000000001e-05 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:9799 3.9 e38/24 3:94cabc51:::rbd_data.111e33663d62.000000000000004c:head v 38'54, mlcod=38'50)", 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.289716007, 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020855719999999999, 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504305+0000", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504306+0000", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7000000000000003e-07 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504313+0000", 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5849999999999996e-06 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504322+0000", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.397e-06 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504327+0000", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7720000000000002e-06 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504378+0000", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0846000000000001e-05 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.504409+0000", 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0876000000000001e-05 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.506344+0000", 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019355069999999999 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.506391+0000", 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6619000000000002e-05 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:13098 3.4 e44/24 3:2785aae7:::rbd_data.111e4c6c7d62.0000000000000313:head v 44'95, mlcod=39'93)", 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:35.652795+0000", 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.141226729, 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018673800000000001, 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652795+0000", 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652795+0000", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652796+0000", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.113e-06 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652811+0000", 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5270000000000001e-05 2024-09-21T22:50:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652831+0000", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9758999999999998e-05 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652841+0000", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0643e-05 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652929+0000", 2024-09-21T22:50:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7459000000000004e-05 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.652950+0000", 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1556000000000002e-05 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.654623+0000", 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001672565 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.654662+0000", 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9014999999999998e-05 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:13204 3.4 e44/24 3:26e77047:::rbd_data.111e4c6c7d62.0000000000000208:head v 44'102, mlcod=44'100)", 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:35.776117+0000", 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.017904389, 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023976539999999999, 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776117+0000", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776117+0000", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776118+0000", 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.46e-06 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776141+0000", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2175e-05 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776161+0000", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0883999999999999e-05 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776171+0000", 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1160000000000005e-06 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776453+0000", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028203499999999998 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776471+0000", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8717999999999999e-05 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.778485+0000", 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020137929999999998 2024-09-21T22:50:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.778515+0000", 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9473000000000001e-05 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:13206 3.9 e44/24 3:965faa2c:::rbd_data.111e4c6c7d62.000000000000020a:head v 44'94, mlcod=44'92)", 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:35.776225+0000", 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.017796464000000001, 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019995759999999999, 2024-09-21T22:50:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776225+0000", 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776225+0000", 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776225+0000", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4099999999999999e-07 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776244+0000", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776252+0000", 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4800000000000001e-06 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776260+0000", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3189999999999995e-06 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776331+0000", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0440999999999999e-05 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.776352+0000", 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1146e-05 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.778179+0000", 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001827146 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:35.778224+0000", 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5259999999999997e-05 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:35.880 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:35.880 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:35.793+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:35.880 INFO:teuthology.orchestra.run.smithi094.stdout:2237 trunc from 0xdcfc2a9 to 0x3054cdb 2024-09-21T22:50:35.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:2238 read 0x2787ac7 thru 0x278d754 (0x5c8e bytes) 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:2242 trunc from 0x3054cdb to 0xd48fbda 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:2243 punch from 0x3b6e6bc to 0x3b7b325, (0xcc69 bytes) 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:2244 punch from 0x289c5b0 to 0x28a5b09, (0x9559 bytes) 2024-09-21T22:50:35.882 INFO:teuthology.orchestra.run.smithi094.stdout:2245 write 0x7508efb thru 0x7517c9e (0xeda4 bytes) 2024-09-21T22:50:35.888 INFO:teuthology.orchestra.run.smithi094.stdout:2246 read 0xe34349 thru 0xe3fa41 (0xb6f9 bytes) 2024-09-21T22:50:35.888 INFO:teuthology.orchestra.run.smithi094.stdout:2247 write 0x6a99820 thru 0x6aa4133 (0xa914 bytes) 2024-09-21T22:50:35.889 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:35.889 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:2248 write 0xdcd8709 thru 0xdcd99bf (0x12b7 bytes) 2024-09-21T22:50:35.911 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:35.905+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:35.917 INFO:teuthology.orchestra.run.smithi094.stdout:2250 punch from 0x67545ec to 0x675575b, (0x116f bytes) 2024-09-21T22:50:35.917 INFO:teuthology.orchestra.run.smithi094.stdout:2252 write 0x9ad0e1b thru 0x9ad3eea (0x30d0 bytes) 2024-09-21T22:50:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:2254 punch from 0x49fc4b to 0x4aac83, (0xb038 bytes) 2024-09-21T22:50:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:2255 write 0x8c9bc49 thru 0x8c9fa6f (0x3e27 bytes) 2024-09-21T22:50:35.932 INFO:teuthology.orchestra.run.smithi094.stdout:2257 write 0x8ba03f4 thru 0x8baa5d3 (0xa1e0 bytes) 2024-09-21T22:50:35.946 INFO:teuthology.orchestra.run.smithi094.stdout:2259 writesame 0x9234c05 thru 0x9242a93 data_size 0x35(0xde8f bytes) 2024-09-21T22:50:35.964 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:35.967 INFO:teuthology.orchestra.run.smithi094.stdout:2262 trunc from 0xdcd99c0 to 0xb87e354 2024-09-21T22:50:35.979 INFO:teuthology.orchestra.run.smithi094.stdout:2265 punch from 0x4ac4c04 to 0x4accc2d, (0x8029 bytes) 2024-09-21T22:50:35.980 INFO:teuthology.orchestra.run.smithi094.stdout:2266 read 0x142afdf thru 0x143af1e (0xff40 bytes) 2024-09-21T22:50:35.980 INFO:teuthology.orchestra.run.smithi094.stdout:2267 write 0x78c548d thru 0x78d1dc2 (0xc936 bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2268 punch from 0x1eff374 to 0x1f016d5, (0x2361 bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2270 read 0x22464d2 thru 0x2248d50 (0x287f bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2272 read 0xb812e5b thru 0xb819d9e (0x6f44 bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2273 punch from 0x185c6f1 to 0x1861d14, (0x5623 bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2274 read 0xa533b57 thru 0xa534f53 (0x13fd bytes) 2024-09-21T22:50:35.997 INFO:teuthology.orchestra.run.smithi094.stdout:2276 punch from 0x4c0a4b8 to 0x4c111f6, (0x6d3e bytes) 2024-09-21T22:50:35.998 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:35.997+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:36.006 INFO:teuthology.orchestra.run.smithi094.stdout:2278 trunc from 0xb87e354 to 0x528ddf6 2024-09-21T22:50:36.011 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13504 3.8 3:17416bf0:::rbd_data.111e4c6c7d62.00000000000001a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.009532+0000", 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00139303, 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001420618, 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13504 2024-09-21T22:50:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009532+0000", 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009532+0000", 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009534+0000", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009542+0000", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.746e-06 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009561+0000", 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8484000000000001e-05 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009571+0000", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.007e-05 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009644+0000", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3844999999999995e-05 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009742+0000", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7181000000000003e-05 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009861+0000", 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119848 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13505 3.b 3:d3bce80d:::rbd_data.111e4c6c7d62.00000000000001a9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.009600+0000", 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00132552, 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001432575, 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13505 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009600+0000", 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009600+0000", 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009600+0000", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4799999999999998e-07 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009606+0000", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9270000000000002e-06 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009614+0000", 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.605e-06 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009619+0000", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1290000000000004e-06 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009702+0000", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2712000000000004e-05 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009848+0000", 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014610900000000001 2024-09-21T22:50:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.010036+0000", 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018814400000000001 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13506 3.7 3:e56d38a7:::rbd_data.111e4c6c7d62.00000000000001aa:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.009635+0000", 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001290191, 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001478956, 2024-09-21T22:50:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13506 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009635+0000", 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009635+0000", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009636+0000", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.75e-07 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009638+0000", 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9069999999999999e-06 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009646+0000", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3629999999999996e-06 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009651+0000", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4369999999999998e-06 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009745+0000", 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3603999999999994e-05 2024-09-21T22:50:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009823+0000", 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8172999999999998e-05 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009930+0000", 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106696 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13508 3.7 3:e2560d0f:::rbd_data.111e4c6c7d62.00000000000001ac:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.009668+0000", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001257546, 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015199549999999999, 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13508 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009668+0000", 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009668+0000", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009668+0000", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3099999999999998e-07 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009671+0000", 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.954e-06 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009679+0000", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5460000000000003e-06 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.009685+0000", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8499999999999999e-06 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.010279+0000", 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00059445600000000004 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.010351+0000", 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2113999999999994e-05 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.010449+0000", 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7517999999999997e-05 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 4 2024-09-21T22:50:36.028 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:50:36.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:36.133 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:36.129+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:36.190 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:36.190 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:36.199 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:36.193+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:36.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:36.300 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:36.297+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:619 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.688759884, 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013060782, 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 619 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673456+0000", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673457+0000", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.58e-07 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673459+0000", 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6539999999999999e-06 2024-09-21T22:50:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673465+0000", 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.02e-06 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673470+0000", 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4479999999999996e-06 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682015+0000", 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0085449849999999997 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.682193+0000", 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017775199999999999 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.683339+0000", 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011463549999999999 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686337+0000", 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00299797 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686383+0000", 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6462999999999999e-05 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686467+0000", 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3796999999999997e-05 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686517+0000", 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9879999999999997e-05 2024-09-21T22:50:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:620 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.688740436, 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014257994, 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 620 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673479+0000", 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5560000000000002e-06 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673495+0000", 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.615e-05 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673500+0000", 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9439999999999996e-06 2024-09-21T22:50:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684881+0000", 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011381242 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684990+0000", 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109197 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685289+0000", 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029849100000000001 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687532+0000", 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022436510000000002 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687588+0000", 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5655999999999997e-05 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687652+0000", 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3544e-05 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687734+0000", 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2126999999999995e-05 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:621 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.688709771999999, 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018613351, 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 621 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673507+0000", 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673510+0000", 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8059999999999999e-06 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673516+0000", 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3099999999999997e-06 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673520+0000", 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4619999999999998e-06 2024-09-21T22:50:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686563+0000", 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013042876 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686663+0000", 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9570999999999999e-05 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687059+0000", 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039618800000000001 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687883+0000", 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082348899999999995 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692061+0000", 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004178022 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692094+0000", 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3473999999999998e-05 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692120+0000", 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5711000000000002e-05 2024-09-21T22:50:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:680 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.616033409, 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013778161000000001, 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 680 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746187+0000", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9160000000000003e-06 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746193+0000", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746200+0000", 2024-09-21T22:50:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5760000000000002e-06 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753332+0000", 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071319840000000001 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753409+0000", 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7111000000000001e-05 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753698+0000", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028885599999999999 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754451+0000", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075351400000000005 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759904+0000", 2024-09-21T22:50:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054529679999999999 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759937+0000", 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3018999999999997e-05 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759961+0000", 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3889000000000001e-05 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1046 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178913758, 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015794288, 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1046 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183304+0000", 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1930000000000001e-06 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183311+0000", 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3339999999999996e-06 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183339+0000", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8030000000000001e-05 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183346+0000", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.192825+0000", 2024-09-21T22:50:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094795499999999998 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193017+0000", 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000192287 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193797+0000", 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000779388 2024-09-21T22:50:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197391+0000", 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035944969999999999 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198914+0000", 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015223039999999999 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199027+0000", 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113127 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199097+0000", 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0024999999999994e-05 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1047 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178861133000002, 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017348148000000001, 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1047 2024-09-21T22:50:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183356+0000", 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183362+0000", 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.806e-06 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183371+0000", 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5529999999999998e-06 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183378+0000", 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8689999999999998e-06 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194941+0000", 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011562767999999999 2024-09-21T22:50:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195056+0000", 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115281 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195493+0000", 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043665799999999998 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199120+0000", 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036272919999999998 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200347+0000", 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001226782 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200566+0000", 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021947500000000001 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200703+0000", 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013704200000000001 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1048 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178828807999999, 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017543792999999998, 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1048 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183388+0000", 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183392+0000", 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7139999999999999e-06 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183401+0000", 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2469999999999997e-06 2024-09-21T22:50:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183408+0000", 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0469999999999996e-06 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195923+0000", 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012514658 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196021+0000", 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8113000000000001e-05 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196394+0000", 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037272199999999997 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200738+0000", 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043443179999999998 2024-09-21T22:50:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200803+0000", 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5166000000000006e-05 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200869+0000", 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6190000000000002e-05 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200931+0000", 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1996999999999996e-05 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1049 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178799173000002, 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017955736999999999, 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1049 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183418+0000", 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183421+0000", 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5580000000000001e-06 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183431+0000", 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3430000000000002e-06 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183438+0000", 2024-09-21T22:50:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197463+0000", 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014025892999999999 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197596+0000", 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013223600000000001 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198242+0000", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064663999999999998 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201081+0000", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002839144 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201120+0000", 2024-09-21T22:50:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8563000000000003e-05 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201262+0000", 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014153000000000001 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201373+0000", 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111236 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1050 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178769313, 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020333185, 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1050 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183448+0000", 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183451+0000", 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6389999999999998e-06 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183460+0000", 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183467+0000", 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9600000000000003e-06 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199154+0000", 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015686734000000001 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199259+0000", 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105364 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199748+0000", 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048820500000000002 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203483+0000", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003735787 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203519+0000", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5954000000000002e-05 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203653+0000", 2024-09-21T22:50:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013328799999999999 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203780+0000", 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012739399999999999 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1051 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178715930999999, 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021843896000000002, 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1051 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183501+0000", 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2399999999999997e-07 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183506+0000", 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183527+0000", 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1353e-05 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183535+0000", 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0180000000000001e-06 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201414+0000", 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01787888 2024-09-21T22:50:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201496+0000", 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2201999999999997e-05 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201824+0000", 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032739100000000001 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205098+0000", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032737280000000001 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205130+0000", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2805000000000002e-05 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205258+0000", 2024-09-21T22:50:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127911 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205344+0000", 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5882000000000002e-05 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1052 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178668105, 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022039640999999999, 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1052 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183549+0000", 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183553+0000", 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1130000000000003e-06 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183563+0000", 2024-09-21T22:50:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6930000000000002e-06 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183570+0000", 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202302+0000", 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018731863000000001 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202413+0000", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111117 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202922+0000", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050897300000000002 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205382+0000", 2024-09-21T22:50:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024604240000000001 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205403+0000", 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1058999999999999e-05 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205503+0000", 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100068 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205588+0000", 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4584000000000002e-05 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1053 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.178637015, 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022880587000000001, 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1053 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183580+0000", 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183592+0000", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.224e-05 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183602+0000", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094e-05 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183609+0000", 2024-09-21T22:50:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1810000000000004e-06 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203845+0000", 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020235907000000001 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203896+0000", 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1187999999999998e-05 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204112+0000", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021558999999999999 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205609+0000", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001496743 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206370+0000", 2024-09-21T22:50:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076130800000000004 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206421+0000", 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0491000000000002e-05 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206460+0000", 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9218999999999998e-05 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1054 3.2 3:46641ad1:::rbd_data.111ecbf88771.0000000000000000:head [stat,truncate 204800] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.174440204, 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019736885999999999, 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1054 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187777+0000", 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187783+0000", 2024-09-21T22:50:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2780000000000001e-06 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187794+0000", 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1464999999999999e-05 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187811+0000", 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6716999999999999e-05 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204451+0000", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016639856000000001 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204504+0000", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3763e-05 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204604+0000", 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9433999999999997e-05 2024-09-21T22:50:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205621+0000", 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010175049999999999 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207340+0000", 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718884 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207409+0000", 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8959000000000002e-05 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207513+0000", 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010366600000000001 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1805 3.b 3:d5e4ffc8:::rbd_data.111ea6ec99f1.000000000000009d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.303475018, 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.032681931999999997, 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1805 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058743+0000", 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.37e-06 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058751+0000", 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058769+0000", 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7927e-05 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058780+0000", 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1353e-05 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058934+0000", 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015403899999999999 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.059298+0000", 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036453700000000001 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065146+0000", 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058479789999999997 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091319+0000", 2024-09-21T22:50:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026172104000000002 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091334+0000", 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4939999999999999e-05 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091379+0000", 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5593000000000002e-05 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091423+0000", 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4014000000000002e-05 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3479 3.7 3:e3964c56:::rbd_data.111e7572a6e.0000000000000000:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.168593819000002, 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01465428, 2024-09-21T22:50:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3479 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193624+0000", 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9199999999999998e-06 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193634+0000", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0132999999999999e-05 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193666+0000", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1405999999999997e-05 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193684+0000", 2024-09-21T22:50:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8185999999999998e-05 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193764+0000", 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0051999999999999e-05 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193901+0000", 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013736 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.208277+0000", 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014375222999999999 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3487 3.b 3:d69a8a6f:::rbd_data.111e7572a6e.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.167653164000001, 2024-09-21T22:50:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01338204, 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3487 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194564+0000", 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4470000000000001e-06 2024-09-21T22:50:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194573+0000", 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-06 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194596+0000", 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3312999999999999e-05 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194611+0000", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194681+0000", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9865999999999998e-05 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194803+0000", 2024-09-21T22:50:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012176900000000001 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.207945+0000", 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013141972 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6462 3.2 3:44ccee00:::rbd_data.111ebc806776.000000000000001a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.600044028999999, 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.08074365, 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6462 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762174+0000", 2024-09-21T22:50:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3029999999999998e-06 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762186+0000", 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.136e-05 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762217+0000", 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0663000000000001e-05 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762232+0000", 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5099999999999999e-05 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762388+0000", 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000156362 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762632+0000", 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243995 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.777316+0000", 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014684262999999999 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842690+0000", 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065373938000000006 2024-09-21T22:50:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842722+0000", 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2283999999999998e-05 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842825+0000", 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102192 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842916+0000", 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1189999999999999e-05 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:6462 3.2 e34/24 ondisk, result = 0)", 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.540630127, 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021121876000000001, 2024-09-21T22:50:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821588+0000", 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0600000000000002e-06 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821598+0000", 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8160000000000005e-06 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821633+0000", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4975999999999999e-05 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821651+0000", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8287e-05 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842659+0000", 2024-09-21T22:50:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021007426999999999 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842677+0000", 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8533e-05 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842708+0000", 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0777e-05 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:9376 3.2 3:4315e470:::rbd_data.111e33663d62.0000000000000004:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.9453210060000004, 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013487305, 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 9376 2024-09-21T22:50:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416896+0000", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8500000000000001e-07 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416900+0000", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7450000000000001e-06 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416907+0000", 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7360000000000001e-06 2024-09-21T22:50:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416913+0000", 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.418988+0000", 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020751480000000002 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419086+0000", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7659999999999997e-05 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419282+0000", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019622899999999999 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.420737+0000", 2024-09-21T22:50:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455582 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430236+0000", 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094984900000000001 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430316+0000", 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0382000000000004e-05 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430383+0000", 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6178000000000002e-05 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:11367 3.1e 3:7cc77dab:::rbd_data.111ea12b32db.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.5477466959999999, 2024-09-21T22:50:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044095079000000002, 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11367 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814472+0000", 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0930000000000001e-06 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814483+0000", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1293e-05 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814512+0000", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9373000000000002e-05 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814527+0000", 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4589999999999999e-05 2024-09-21T22:50:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814641+0000", 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114339 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814787+0000", 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014541900000000001 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858463+0000", 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043676404000000002 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858508+0000", 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4731000000000003e-05 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858565+0000", 2024-09-21T22:50:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6836999999999998e-05 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:36.496 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:36.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:36.498 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:36.493+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:36.599 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:36.597+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:36.631 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:36.631 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:36.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:36.700 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:36.693+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:36.762 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:36.762 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:36.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:36.800 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:36.793+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:36.901 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:36.897+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:36.905 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:11225 3.1d e40/24 3:bfcb9502:::rbd_data.111ea12b32db.0000000000000000:head v 40'541, mlcod=38'539)", 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.3874580769999998, 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020663840999999999, 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514418+0000", 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2960000000000001e-06 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518906+0000", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044876899999999999 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518949+0000", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2960000000000002e-05 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518968+0000", 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9395999999999999e-05 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.519165+0000", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019721300000000001 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.519210+0000", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4419999999999998e-05 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535001+0000", 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015791052 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535080+0000", 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8813999999999995e-05 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11629 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0644945129999996, 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017211015999999999, 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11629 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837380+0000", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8599999999999998e-07 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837384+0000", 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2629999999999997e-06 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837394+0000", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8300000000000008e-06 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837405+0000", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0779e-05 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.844652+0000", 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072473060000000002 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.844915+0000", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026219600000000002 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.846043+0000", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001128275 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.849721+0000", 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036786119999999999 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854498+0000", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047762430000000003 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854551+0000", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3158999999999997e-05 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854590+0000", 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666999999999998e-05 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11630 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0644590760000003, 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018432139, 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11630 2024-09-21T22:50:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837416+0000", 2024-09-21T22:50:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837420+0000", 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2259999999999999e-06 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837429+0000", 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4460000000000007e-06 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837437+0000", 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4699999999999996e-06 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.847532+0000", 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010094893000000001 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.847688+0000", 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000156603 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.848375+0000", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068659400000000001 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852052+0000", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036775219999999999 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855725+0000", 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036722859999999999 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855797+0000", 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2878000000000003e-05 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855847+0000", 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9570999999999997e-05 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11631 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0644277549999996, 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018496809999999999, 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11631 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837447+0000", 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837451+0000", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9029999999999997e-06 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837460+0000", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2960000000000004e-06 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837467+0000", 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2169999999999997e-06 2024-09-21T22:50:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.849884+0000", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012417167999999999 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.850109+0000", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000224582 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.851056+0000", 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094672699999999996 2024-09-21T22:50:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854725+0000", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036696279999999999 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855857+0000", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011317829999999999 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855898+0000", 2024-09-21T22:50:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1182e-05 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855943+0000", 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4684000000000001e-05 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11632 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0643974170000003, 2024-09-21T22:50:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020031846999999998, 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11632 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3e-07 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837481+0000", 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4570000000000001e-06 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837493+0000", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2702e-05 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837501+0000", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852143+0000", 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014642654 2024-09-21T22:50:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852358+0000", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000214985 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.853448+0000", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010893680000000001 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856050+0000", 2024-09-21T22:50:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026024759999999998 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857364+0000", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013136210000000001 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857442+0000", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8516999999999994e-05 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857508+0000", 2024-09-21T22:50:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6230999999999997e-05 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11633 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0643639260000004, 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020634830999999999, 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11633 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837511+0000", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.44e-07 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837515+0000", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8299999999999998e-06 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837524+0000", 2024-09-21T22:50:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1889999999999996e-06 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837531+0000", 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1760000000000004e-06 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854772+0000", 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017241480999999999 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854844+0000", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1320000000000002e-05 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855136+0000", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029208499999999998 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857531+0000", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023956279999999999 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857848+0000", 2024-09-21T22:50:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031616400000000002 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858024+0000", 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017639500000000001 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858145+0000", 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120919 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11634 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0643338340000001, 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023090243, 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11634 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837541+0000", 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1999999999999999e-07 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837545+0000", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0500000000000002e-06 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837554+0000", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2450000000000004e-06 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837561+0000", 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3830000000000003e-06 2024-09-21T22:50:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856110+0000", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018548565999999999 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856220+0000", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110226 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856770+0000", 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055009000000000004 2024-09-21T22:50:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858496+0000", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001725578 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860323+0000", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018268080000000001 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860498+0000", 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175582 2024-09-21T22:50:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860630+0000", 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013209499999999999 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11649 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865868+0000", 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0360064280000003, 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015336897, 2024-09-21T22:50:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11649 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865868+0000", 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865868+0000", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865868+0000", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865871+0000", 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5600000000000001e-06 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865876+0000", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3650000000000003e-06 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865881+0000", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5070000000000002e-06 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.875050+0000", 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091692939999999997 2024-09-21T22:50:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.875153+0000", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103479 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.875497+0000", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000344058 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.880889+0000", 2024-09-21T22:50:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053913889999999999 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.880933+0000", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4419999999999998e-05 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881081+0000", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014781399999999999 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881204+0000", 2024-09-21T22:50:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123375 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11650 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0359849270000003, 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019674952999999998, 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11650 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9400000000000001e-07 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865892+0000", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.802e-06 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865898+0000", 2024-09-21T22:50:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5079999999999996e-06 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865902+0000", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6779999999999999e-06 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.876555+0000", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010652134000000001 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.876799+0000", 2024-09-21T22:50:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024472799999999999 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.877868+0000", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010684049999999999 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881258+0000", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033901510000000001 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885471+0000", 2024-09-21T22:50:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042133170000000003 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885520+0000", 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8467999999999997e-05 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885564+0000", 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4367999999999999e-05 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11651 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0359654069999999, 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019787837999999999, 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11651 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.65e-07 2024-09-21T22:50:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865912+0000", 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6350000000000002e-06 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865917+0000", 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3369999999999999e-06 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865921+0000", 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.879328+0000", 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013407117 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.879489+0000", 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160278 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.880085+0000", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059590499999999998 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881376+0000", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001291798 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885586+0000", 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00420959 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885657+0000", 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1483999999999995e-05 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885696+0000", 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9053e-05 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11652 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0359468449999998, 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019872035999999999, 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11652 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865928+0000", 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7800000000000002e-07 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865930+0000", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5019999999999999e-06 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865935+0000", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3179999999999997e-06 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865940+0000", 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1540000000000004e-06 2024-09-21T22:50:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881478+0000", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015538847999999999 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881623+0000", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014447 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882055+0000", 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043187100000000001 2024-09-21T22:50:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885404+0000", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033497190000000001 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885703+0000", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029899599999999999 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885764+0000", 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0492000000000003e-05 2024-09-21T22:50:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885799+0000", 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5287999999999998e-05 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11653 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0359250510000004, 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019953525999999999, 2024-09-21T22:50:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11653 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865952+0000", 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6850000000000001e-06 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865958+0000", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.857e-06 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865962+0000", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3490000000000002e-06 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882603+0000", 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016640818000000002 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882697+0000", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4203000000000006e-05 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.883711+0000", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001013835 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885460+0000", 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017489579999999999 2024-09-21T22:50:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885806+0000", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000346168 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885849+0000", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.295e-05 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885902+0000", 2024-09-21T22:50:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3266999999999997e-05 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11654 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0359057810000003, 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021916723999999999, 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11654 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865969+0000", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.65e-07 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865971+0000", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6350000000000002e-06 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865976+0000", 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1970000000000004e-06 2024-09-21T22:50:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865981+0000", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5480000000000003e-06 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884316+0000", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018334613 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884425+0000", 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010932 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884873+0000", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00044802900000000002 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885970+0000", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010975410000000001 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887655+0000", 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001685001 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887806+0000", 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150536 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887885+0000", 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8938999999999998e-05 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11811 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.015142+0000", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8867322390000001, 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015072230000000001, 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11811 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015142+0000", 2024-09-21T22:50:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015142+0000", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015142+0000", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2399999999999999e-07 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015145+0000", 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4770000000000002e-06 2024-09-21T22:50:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015151+0000", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9710000000000001e-06 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015155+0000", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7609999999999998e-06 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.026316+0000", 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011160728 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.026405+0000", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8857999999999995e-05 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.027040+0000", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063511199999999996 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.028650+0000", 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016098250000000001 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.029976+0000", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001326034 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030114+0000", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013761399999999999 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030214+0000", 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100426 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11812 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8867105400000002, 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015836000999999999, 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11812 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015164+0000", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.08e-07 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015167+0000", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.993e-06 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015173+0000", 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015179+0000", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1490000000000002e-06 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.028702+0000", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013523599000000001 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.028783+0000", 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0817999999999998e-05 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.029222+0000", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043925399999999998 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030263+0000", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010406809999999999 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030944+0000", 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068143100000000003 2024-09-21T22:50:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030976+0000", 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1547000000000001e-05 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030999+0000", 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3567999999999999e-05 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13361 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870095+0000", 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0317789509999999, 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013987893, 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13361 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870095+0000", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870095+0000", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870095+0000", 2024-09-21T22:50:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4799999999999999e-07 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870099+0000", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0989999999999999e-06 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870106+0000", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0809999999999996e-06 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870111+0000", 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1449999999999999e-06 2024-09-21T22:50:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.880127+0000", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010016647 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.880244+0000", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116161 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.880835+0000", 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059127799999999999 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.882944+0000", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021089440000000002 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.883978+0000", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010337569999999999 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884039+0000", 2024-09-21T22:50:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1624999999999994e-05 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884083+0000", 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3708000000000002e-05 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13362 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870118+0000", 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.031755805, 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015607515000000001, 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13362 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870118+0000", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870118+0000", 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870119+0000", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0699999999999998e-07 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870122+0000", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870128+0000", 2024-09-21T22:50:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.737e-06 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870134+0000", 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6799999999999998e-06 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.881958+0000", 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011824061 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.882059+0000", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010033400000000001 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.882443+0000", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038411599999999999 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885499+0000", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030561920000000001 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885541+0000", 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1857999999999998e-05 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885671+0000", 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013029200000000001 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885726+0000", 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4728000000000002e-05 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13363 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.031732235, 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016584944000000001, 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13363 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000001e-07 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870145+0000", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870153+0000", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870158+0000", 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.045e-06 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.882999+0000", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012840356000000001 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.883090+0000", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1393999999999999e-05 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.883490+0000", 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039968000000000002 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885780+0000", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002290566 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886506+0000", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072612999999999996 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886622+0000", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115968 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886727+0000", 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104251 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13364 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.031707774, 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016765598, 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13364 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870167+0000", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6100000000000001e-07 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870179+0000", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2513e-05 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870187+0000", 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7789999999999996e-06 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870192+0000", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3870000000000003e-06 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884122+0000", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013930120000000001 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884207+0000", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4359999999999999e-05 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884627+0000", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042007699999999998 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885953+0000", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013261049999999999 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886750+0000", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00079673700000000005 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886835+0000", 2024-09-21T22:50:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5037000000000006e-05 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886932+0000", 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7021999999999994e-05 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13488 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.981104+0000", 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.920769897, 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014169717, 2024-09-21T22:50:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13488 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981104+0000", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981104+0000", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981105+0000", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981107+0000", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.734e-06 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981115+0000", 2024-09-21T22:50:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.055e-06 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.981121+0000", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2399999999999998e-06 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.993563+0000", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012442507 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.993628+0000", 2024-09-21T22:50:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4559000000000003e-05 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.993885+0000", 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025716600000000002 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.994708+0000", 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082364000000000005 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.995181+0000", 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047257499999999998 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.995233+0000", 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170999999999999e-05 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.995274+0000", 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0574000000000002e-05 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:37.018 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:37.013+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:37.057 INFO:teuthology.orchestra.run.smithi094.stdout:2279 read 0x3cca4f0 thru 0x3cd497f (0xa490 bytes) 2024-09-21T22:50:37.081 INFO:teuthology.orchestra.run.smithi094.stdout:2280 trunc from 0x528ddf6 to 0xe715f66 2024-09-21T22:50:37.093 INFO:teuthology.orchestra.run.smithi094.stdout:2281 write 0x86282e9 thru 0x862f6e2 (0x73fa bytes) 2024-09-21T22:50:37.107 INFO:teuthology.orchestra.run.smithi094.stdout:2282 read 0xdc76f41 thru 0xdc778f6 (0x9b6 bytes) 2024-09-21T22:50:37.107 INFO:teuthology.orchestra.run.smithi094.stdout:2283 writesame 0x8741e9e thru 0x87452ee data_size 0x1(0x3451 bytes) 2024-09-21T22:50:37.120 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:37.113+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:37.123 INFO:teuthology.orchestra.run.smithi094.stdout:2284 read 0x66068a6 thru 0x660cc31 (0x638c bytes) 2024-09-21T22:50:37.134 INFO:teuthology.orchestra.run.smithi094.stdout:2285 trunc from 0xe715f66 to 0xa17b2b0 2024-09-21T22:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout:2288 writesame 0x1ade06e thru 0x1ae8a23 data_size 0x27(0xa9b6 bytes) 2024-09-21T22:50:37.154 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:37.154 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.172 INFO:teuthology.orchestra.run.smithi094.stdout:2289 write 0x218d973 thru 0x2199a5f (0xc0ed bytes) 2024-09-21T22:50:37.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:37.192 INFO:teuthology.orchestra.run.smithi094.stdout:2290 writesame 0x866dd3d thru 0x8675fdf data_size 0x1(0x82a3 bytes) 2024-09-21T22:50:37.203 INFO:teuthology.orchestra.run.smithi094.stdout:2291 read 0x1078cc5 thru 0x107b681 (0x29bd bytes) 2024-09-21T22:50:37.211 INFO:teuthology.orchestra.run.smithi094.stdout:2293 trunc from 0xa17b2b0 to 0xcfa47c5 2024-09-21T22:50:37.219 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:37.217+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:37.220 INFO:teuthology.orchestra.run.smithi094.stdout:2295 write 0xadcfa43 thru 0xaddf58e (0xfb4c bytes) 2024-09-21T22:50:37.242 INFO:teuthology.orchestra.run.smithi094.stdout:2298 punch from 0x9cc47ee to 0x9cd3c30, (0xf442 bytes) 2024-09-21T22:50:37.242 INFO:teuthology.orchestra.run.smithi094.stdout:2299 write 0xb7ec56 thru 0xb7f052 (0x3fd bytes) 2024-09-21T22:50:37.245 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:37.245 INFO:teuthology.orchestra.run.smithi094.stdout:2303 punch from 0xb3ba9c8 to 0xb3c1208, (0x6840 bytes) 2024-09-21T22:50:37.245 INFO:teuthology.orchestra.run.smithi094.stdout:2305 read 0x30fd99a thru 0x3101206 (0x386d bytes) 2024-09-21T22:50:37.246 INFO:teuthology.orchestra.run.smithi094.stdout:2306 write 0x7c8697e thru 0x7c92eba (0xc53d bytes) 2024-09-21T22:50:37.267 INFO:teuthology.orchestra.run.smithi094.stdout:2307 read 0x2a12b81 thru 0x2a18b14 (0x5f94 bytes) 2024-09-21T22:50:37.283 INFO:teuthology.orchestra.run.smithi094.stdout:2308 trunc from 0xcfa47c5 to 0x686bd33 2024-09-21T22:50:37.309 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:37.309 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.314 INFO:teuthology.orchestra.run.smithi094.stdout:2312 clone 11 order 20 su 16384 sc 3 2024-09-21T22:50:37.320 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:37.313+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:37.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:37.421 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:37.417+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9308 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0941053350000001, 2024-09-21T22:50:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030613436000000001, 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9308 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391672+0000", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391681+0000", 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3389999999999999e-06 2024-09-21T22:50:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391700+0000", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8725000000000001e-05 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391714+0000", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3839000000000001e-05 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.421988+0000", 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03027432 2024-09-21T22:50:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422137+0000", 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014888499999999999 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422284+0000", 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014702500000000001 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9309 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0940416899999992, 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030903685, 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9309 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391736+0000", 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2729999999999999e-06 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391741+0000", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9100000000000002e-06 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391761+0000", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9565e-05 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391775+0000", 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3596e-05 2024-09-21T22:50:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422343+0000", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030568828999999999 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422495+0000", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015208600000000001 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422638+0000", 2024-09-21T22:50:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014242599999999999 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9319 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0937376289999996, 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030911366999999999, 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9319 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392040+0000", 2024-09-21T22:50:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.553e-06 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392046+0000", 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1360000000000002e-06 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392065+0000", 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392079+0000", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3815000000000001e-05 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422311+0000", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030232335999999999 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422480+0000", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000169331 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422950+0000", 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00046917699999999998 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9320 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0936765370000003, 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031731672000000002, 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9320 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392100+0000", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0470000000000001e-06 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392118+0000", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7184999999999999e-05 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392138+0000", 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0795e-05 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392153+0000", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4309e-05 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423024+0000", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030871742000000001 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423635+0000", 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061065999999999996 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423831+0000", 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019593400000000001 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9321 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0936026430000005, 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032090151999999997, 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9321 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392175+0000", 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-06 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392182+0000", 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.746e-06 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392201+0000", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8402000000000001e-05 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392209+0000", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.636e-06 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423902+0000", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031692531000000003 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424088+0000", 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018566599999999999 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424263+0000", 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017582399999999999 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9328 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 98304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0935550430000003, 2024-09-21T22:50:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031544833000000001, 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9328 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392222+0000", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1399999999999996e-07 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392225+0000", 2024-09-21T22:50:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5389999999999999e-06 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392235+0000", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0003000000000001e-05 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392243+0000", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.542e-06 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422696+0000", 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030453424 2024-09-21T22:50:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422834+0000", 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137539 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423766+0000", 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093207200000000004 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9329 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 425984~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.093523502, 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031920959999999998, 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9329 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392253+0000", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392256+0000", 2024-09-21T22:50:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2729999999999998e-06 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392266+0000", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3700000000000001e-06 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392273+0000", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3459999999999996e-06 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423823+0000", 2024-09-21T22:50:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031550337999999997 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423991+0000", 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000167932 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424173+0000", 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000182051 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9330 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0934936709999992, 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032524491000000003, 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9330 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392283+0000", 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392286+0000", 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.864e-06 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392299+0000", 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3382e-05 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392306+0000", 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2760000000000003e-06 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424267+0000", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031960323999999998 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424448+0000", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181423 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424807+0000", 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035859899999999997 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9331 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0934603500000009, 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033079164000000001, 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9331 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5400000000000001e-07 2024-09-21T22:50:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392319+0000", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9359999999999999e-06 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392329+0000", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3829999999999998e-06 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392339+0000", 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424890+0000", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032550548999999998 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425181+0000", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029145400000000002 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425395+0000", 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000213573 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0934275020000008, 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0933840719999992, 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0932754800000009, 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9341 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0932148379999997, 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032779214000000001, 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9341 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392562+0000", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0240000000000001e-06 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392568+0000", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3990000000000003e-06 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392585+0000", 2024-09-21T22:50:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7125e-05 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392599+0000", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3946e-05 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424377+0000", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031778433000000002 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424536+0000", 2024-09-21T22:50:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159304 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425340+0000", 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080398299999999998 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9342 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 557056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0931590520000007, 2024-09-21T22:50:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033794164000000002, 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9342 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392618+0000", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392623+0000", 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-21T22:50:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392641+0000", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7805999999999999e-05 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392654+0000", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3259e-05 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425439+0000", 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032784424999999999 2024-09-21T22:50:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425626+0000", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018740799999999999 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426411+0000", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078503999999999998 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9343 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0931034490000009, 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034396423000000002, 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9343 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392674+0000", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392679+0000", 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6629999999999998e-06 2024-09-21T22:50:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392697+0000", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7518999999999999e-05 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392714+0000", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668e-05 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426525+0000", 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033811433000000002 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426759+0000", 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023436299999999999 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427069+0000", 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000309594 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9344 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0930204959999994, 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034732139000000002, 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9344 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392757+0000", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4929999999999999e-06 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392775+0000", 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7733000000000001e-05 2024-09-21T22:50:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392794+0000", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9134999999999999e-05 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392812+0000", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8238000000000001e-05 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427147+0000", 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034335093999999997 2024-09-21T22:50:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427339+0000", 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001917 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427488+0000", 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000148746 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9345 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0929431080000001, 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034982737999999999, 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9345 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392834+0000", 2024-09-21T22:50:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.373e-06 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392840+0000", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9030000000000001e-06 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392861+0000", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0911999999999999e-05 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392874+0000", 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2896e-05 2024-09-21T22:50:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427564+0000", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034689953000000003 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427715+0000", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015096599999999999 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427816+0000", 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100735 2024-09-21T22:50:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9346 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0928837829999996, 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380256999999998, 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9346 2024-09-21T22:50:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392893+0000", 2024-09-21T22:50:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392898+0000", 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1429999999999997e-06 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392918+0000", 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0165000000000001e-05 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392932+0000", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3073e-05 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427888+0000", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034956787000000003 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428089+0000", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020047700000000001 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428272+0000", 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000183575 2024-09-21T22:50:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0839138940000002, 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11225 3.1d 3:bfcb9502:::rbd_data.111ea12b32db.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.9804092749999995, 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031990183999999998, 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11225 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505370+0000", 2024-09-21T22:50:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512142+0000", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067726949999999996 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512177+0000", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5167999999999999e-05 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512196+0000", 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8882e-05 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512384+0000", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018725 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512632+0000", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024841799999999998 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.513884+0000", 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001252038 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535351+0000", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021467211999999999 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537246+0000", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001894864 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537305+0000", 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8570999999999999e-05 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537357+0000", 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2166000000000002e-05 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:37.571 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:37.573 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:37.569+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:37.673 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:37.673+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:37.739 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:37.740 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:37.740 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:37.740 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.754 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:37.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:37.797 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:37.797+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:37.874 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:37.869+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:37.914 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:37.914 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:37.928 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:37.952 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:37.952 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:37.969 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:37.969 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:37.975 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:37.973+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:37.986 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:37.986 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:38.003 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:38.003 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:38.022 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:38.022 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:38.044 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:38.044 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:38.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.115392773000004, 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:38.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12314 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3272704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9764367780000001, 2024-09-21T22:50:38.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048771374999999999, 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12314 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074253+0000", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.58e-07 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074255+0000", 2024-09-21T22:50:38.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6919999999999998e-06 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074263+0000", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6590000000000007e-06 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074269+0000", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0499999999999997e-06 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.122919+0000", 2024-09-21T22:50:38.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048650021000000002 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.122962+0000", 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3146999999999999e-05 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123024+0000", 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1347999999999996e-05 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12315 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3313664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.97641212, 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049012636999999998, 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12315 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077706+0000", 2024-09-21T22:50:38.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034286859999999998 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077722+0000", 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5849e-05 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077735+0000", 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3149000000000001e-05 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077743+0000", 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6629999999999999e-06 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123072+0000", 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045329031999999998 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123161+0000", 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8658000000000004e-05 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123290+0000", 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012860000000000001 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12367 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 2826240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9687257960000002, 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049023430999999999, 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12367 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081964+0000", 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4499999999999998e-07 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081970+0000", 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0229999999999998e-06 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081974+0000", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7419999999999997e-06 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081977+0000", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.021e-06 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130764+0000", 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048786325999999998 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130855+0000", 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1533000000000006e-05 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130987+0000", 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013154099999999999 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12368 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 2867200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9687079389999997, 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049220224999999999, 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12368 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085829+0000", 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003847984 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085841+0000", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1598e-05 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085848+0000", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7200000000000006e-06 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085853+0000", 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8069999999999999e-06 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131033+0000", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045179929000000001 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131104+0000", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1101999999999998e-05 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131201+0000", 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7084999999999996e-05 2024-09-21T22:50:38.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12383 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3481600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9646426200000002, 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048843009, 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12383 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086047+0000", 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086048+0000", 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086051+0000", 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.343e-06 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086054+0000", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8619999999999999e-06 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134743+0000", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048689112 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134806+0000", 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2904000000000006e-05 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134889+0000", 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3349000000000004e-05 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12384 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3522560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9646315889999997, 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049097057999999999, 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12384 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:38.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089645+0000", 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035875820000000002 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089649+0000", 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089662+0000", 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2323e-05 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089669+0000", 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2559999999999996e-06 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134935+0000", 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045265908000000001 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.135040+0000", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105096 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.135155+0000", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114608 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12447 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3485696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:38.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9526648990000002, 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048875292000000001, 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12447 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:38.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098025+0000", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0499999999999999e-07 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098027+0000", 2024-09-21T22:50:38.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.655e-06 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098043+0000", 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5753999999999999e-05 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098048+0000", 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2739999999999998e-06 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146679+0000", 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048630975 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146785+0000", 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010550100000000001 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146899+0000", 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114728 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12448 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9526332899999996, 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049045176000000003, 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12448 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.092 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9999999999999998e-07 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098059+0000", 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.672e-06 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098066+0000", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-06 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098071+0000", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1590000000000001e-06 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146944+0000", 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048873460000000001 2024-09-21T22:50:38.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147010+0000", 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5673000000000006e-05 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147101+0000", 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0941999999999998e-05 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12449 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3567616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.952611418, 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049294103999999998, 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12449 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.15e-07 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098080+0000", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098087+0000", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2670000000000003e-06 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098091+0000", 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8640000000000004e-06 2024-09-21T22:50:38.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147153+0000", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049061950999999999 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147259+0000", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106026 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147372+0000", 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011244899999999999 2024-09-21T22:50:38.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12450 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3608576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9525910130000002, 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049475004000000003, 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12450 2024-09-21T22:50:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098099+0000", 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6400000000000003e-07 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098101+0000", 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0980000000000001e-06 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098107+0000", 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4799999999999998e-06 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098112+0000", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0170000000000004e-06 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147416+0000", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049304065000000001 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147482+0000", 2024-09-21T22:50:38.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6202000000000001e-05 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147573+0000", 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0678000000000002e-05 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12451 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9525557080000002, 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049716228000000001, 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12451 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:38.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098134+0000", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0600000000000001e-07 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098136+0000", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098140+0000", 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-21T22:50:38.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098144+0000", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2310000000000001e-06 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147626+0000", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049482019000000002 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147732+0000", 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010628600000000001 2024-09-21T22:50:38.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147850+0000", 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117652 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12452 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3690496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9525396820000003, 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049897555000000003, 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12452 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098150+0000", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-07 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098157+0000", 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3749999999999997e-06 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098162+0000", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7520000000000004e-06 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098165+0000", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0599999999999999e-06 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147894+0000", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049729168999999997 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147958+0000", 2024-09-21T22:50:38.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3841000000000001e-05 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148047+0000", 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9090000000000003e-05 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12453 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3731456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9525200600000003, 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050146668999999998, 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12453 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:38.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.03e-07 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098171+0000", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3969999999999999e-06 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098176+0000", 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0139999999999998e-06 2024-09-21T22:50:38.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098179+0000", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148098+0000", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049919193000000001 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148198+0000", 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100271 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148316+0000", 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117296 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12454 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3772416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9525066359999999, 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050328201000000003, 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12454 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098183+0000", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5699999999999999e-07 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098184+0000", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739999999999999e-06 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098187+0000", 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.427e-06 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098190+0000", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0649999999999999e-06 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148360+0000", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050169381999999998 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148425+0000", 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4916e-05 2024-09-21T22:50:38.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148511+0000", 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.598e-05 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12455 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3813376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9524952369999999, 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050678511000000002, 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12455 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5699999999999999e-07 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098195+0000", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.215e-06 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098199+0000", 2024-09-21T22:50:38.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.428e-06 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098202+0000", 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148563+0000", 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050361446999999997 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148670+0000", 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107192 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148872+0000", 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020216400000000001 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12456 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3854336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9524840299999999, 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050867531000000001, 2024-09-21T22:50:38.125 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12456 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5899999999999998e-07 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098207+0000", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.381e-06 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098210+0000", 2024-09-21T22:50:38.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3400000000000002e-06 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098213+0000", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7949999999999999e-06 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148918+0000", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050705061000000003 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148984+0000", 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5808000000000004e-05 2024-09-21T22:50:38.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149073+0000", 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8887000000000005e-05 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12457 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3895296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9524726999999995, 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051131854999999997, 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12457 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101657+0000", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00344014 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101661+0000", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6260000000000003e-06 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101675+0000", 2024-09-21T22:50:38.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3793e-05 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101683+0000", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1629999999999996e-06 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149123+0000", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047440319000000002 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149230+0000", 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010642 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149348+0000", 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011839400000000001 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12463 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 4141056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9488355439999996, 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048991664999999997, 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12463 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3000000000000001e-07 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101856+0000", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.182e-06 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101863+0000", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5429999999999999e-06 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101868+0000", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.023e-06 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150621+0000", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048752789999999997 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150711+0000", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150845+0000", 2024-09-21T22:50:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013396600000000001 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12464 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 4182016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9488140889999999, 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212819999999997, 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12464 2024-09-21T22:50:38.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:38.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8500000000000002e-07 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101882+0000", 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0550000000000002e-06 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101890+0000", 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101897+0000", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7710000000000001e-06 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150884+0000", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048987902 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150971+0000", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6315000000000003e-05 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:38.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.151088+0000", 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011707200000000001 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:38.141 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:38.142 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:38.142 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:38.143 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:38.143 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:38.144 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:38.141+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:38.145 INFO:teuthology.orchestra.run.smithi094.stdout:flattening image image_client.0-clone10 2024-09-21T22:50:38.243 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:38.237+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:38.344 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:38.341+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:38.345 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:38.345 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:38.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:50:38.449 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:38.449+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:38.545 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:38.541+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:38.558 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:38.558 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:38.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:50:38.665 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:38.665+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:38.743 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:38.743 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:38.747 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:38.741+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:38.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T22:50:38.849 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:38.849+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:38.882 INFO:teuthology.orchestra.run.smithi094.stdout:2313 writesame 0xd3f5be2 thru 0xd4058ae data_size 0x1(0xfccd bytes) 2024-09-21T22:50:38.882 INFO:teuthology.orchestra.run.smithi094.stdout:2314 write 0x27b040c thru 0x27b287f (0x2474 bytes) 2024-09-21T22:50:38.888 INFO:teuthology.orchestra.run.smithi094.stdout:2315 write 0xb2a87dd thru 0xb2b57f4 (0xd018 bytes) 2024-09-21T22:50:38.889 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:38.889 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:38.898 INFO:teuthology.orchestra.run.smithi094.stdout:2318 write 0x48d8278 thru 0x48dfaa9 (0x7832 bytes) 2024-09-21T22:50:38.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:38.907 INFO:teuthology.orchestra.run.smithi094.stdout:2319 punch from 0x4201bca to 0x4202464, (0x89a bytes) 2024-09-21T22:50:38.915 INFO:teuthology.orchestra.run.smithi094.stdout:2320 write 0xdaad990 thru 0xdab0037 (0x26a8 bytes) 2024-09-21T22:50:38.922 INFO:teuthology.orchestra.run.smithi094.stdout:2321 read 0xb0b886e thru 0xb0c4bb5 (0xc348 bytes) 2024-09-21T22:50:38.922 INFO:teuthology.orchestra.run.smithi094.stdout:2322 write 0x7950d4c thru 0x79512cf (0x584 bytes) 2024-09-21T22:50:38.928 INFO:teuthology.orchestra.run.smithi094.stdout:2323 write 0xb87de1b thru 0xb87df7e (0x164 bytes) 2024-09-21T22:50:38.933 INFO:teuthology.orchestra.run.smithi094.stdout:2324 punch from 0x6aeed52 to 0x6afa7dc, (0xba8a bytes) 2024-09-21T22:50:38.933 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:38.947 INFO:teuthology.orchestra.run.smithi094.stdout:2326 trunc from 0xdab0038 to 0x40f08cc 2024-09-21T22:50:38.949 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:38.944+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:38.975 INFO:teuthology.orchestra.run.smithi094.stdout:2327 punch from 0x3f36e27 to 0x3f3cc20, (0x5df9 bytes) 2024-09-21T22:50:38.975 INFO:teuthology.orchestra.run.smithi094.stdout:2328 punch from 0x1b43c29 to 0x1b531f5, (0xf5cc bytes) 2024-09-21T22:50:38.976 INFO:teuthology.orchestra.run.smithi094.stdout:2329 read 0x822ebd thru 0x827236 (0x437a bytes) 2024-09-21T22:50:38.988 INFO:teuthology.orchestra.run.smithi094.stdout:2330 write 0x7058da2 thru 0x7061c6a (0x8ec9 bytes) 2024-09-21T22:50:38.997 INFO:teuthology.orchestra.run.smithi094.stdout:2331 writesame 0x4b09b39 thru 0x4b13e37 data_size 0x7(0xa2ff bytes) 2024-09-21T22:50:39.006 INFO:teuthology.orchestra.run.smithi094.stdout:2332 punch from 0x52c7728 to 0x52d4d49, (0xd621 bytes) 2024-09-21T22:50:39.006 INFO:teuthology.orchestra.run.smithi094.stdout:2334 read 0x29c3d1e thru 0x29d0723 (0xca06 bytes) 2024-09-21T22:50:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:2335 punch from 0x9aa208 to 0x9b5c57, (0xba4f bytes) 2024-09-21T22:50:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:2337 read 0x4591fe0 thru 0x459b558 (0x9579 bytes) 2024-09-21T22:50:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:2338 read 0x529c78f thru 0x529e7f6 (0x2068 bytes) 2024-09-21T22:50:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:2339 read 0x3be74dd thru 0x3bedbed (0x6711 bytes) 2024-09-21T22:50:39.009 INFO:teuthology.orchestra.run.smithi094.stdout:2340 trunc from 0x7061c6b to 0x586ccfb 2024-09-21T22:50:39.037 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:50:39.041 INFO:teuthology.orchestra.run.smithi094.stdout:2341 writesame 0x7937e49 thru 0x793f2ef data_size 0x1(0x74a7 bytes) 2024-09-21T22:50:39.041 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.050 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:39.044+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:39.057 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.057 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.058 INFO:teuthology.orchestra.run.smithi094.stdout:2343 write 0xc630d56 thru 0xc63e838 (0xdae3 bytes) 2024-09-21T22:50:39.065 INFO:teuthology.orchestra.run.smithi094.stdout:2344 read 0xc50e33a thru 0xc519841 (0xb508 bytes) 2024-09-21T22:50:39.065 INFO:teuthology.orchestra.run.smithi094.stdout:2345 read 0xb3c23da thru 0xb3cee9c (0xcac3 bytes) 2024-09-21T22:50:39.068 INFO:teuthology.orchestra.run.smithi094.stdout:2347 trunc from 0xc63e839 to 0xe837e30 2024-09-21T22:50:39.076 INFO:teuthology.orchestra.run.smithi094.stdout:2349 read 0x9d0d716 thru 0x9d16ff9 (0x98e4 bytes) 2024-09-21T22:50:39.076 INFO:teuthology.orchestra.run.smithi094.stdout:2350 write 0x71287d3 thru 0x712a96c (0x219a bytes) 2024-09-21T22:50:39.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:39.080 INFO:teuthology.orchestra.run.smithi094.stdout:2352 punch from 0xf270b5 to 0xf3210c, (0xb057 bytes) 2024-09-21T22:50:39.080 INFO:teuthology.orchestra.run.smithi094.stdout:2354 read 0x9a1c372 thru 0x9a23065 (0x6cf4 bytes) 2024-09-21T22:50:39.080 INFO:teuthology.orchestra.run.smithi094.stdout:2356 writesame 0x1c93f97 thru 0x1c99315 data_size 0x5f(0x537f bytes) 2024-09-21T22:50:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:2357 read 0x771873c thru 0x77252c4 (0xcb89 bytes) 2024-09-21T22:50:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:2358 write 0x9a81bfd thru 0x9a8aebd (0x92c1 bytes) 2024-09-21T22:50:39.139 INFO:teuthology.orchestra.run.smithi094.stdout:2359 read 0x6aa2996 thru 0x6aace8e (0xa4f9 bytes) 2024-09-21T22:50:39.139 INFO:teuthology.orchestra.run.smithi094.stdout:2360 writesame 0x980d926 thru 0x9810902 data_size 0x1(0x2fdd bytes) 2024-09-21T22:50:39.151 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:39.144+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:39.156 INFO:teuthology.orchestra.run.smithi094.stdout:2361 read 0xa189ca9 thru 0xa1951eb (0xb543 bytes) 2024-09-21T22:50:39.157 INFO:teuthology.orchestra.run.smithi094.stdout:2362 punch from 0x2cb437a to 0x2cc1e3a, (0xdac0 bytes) 2024-09-21T22:50:39.157 INFO:teuthology.orchestra.run.smithi094.stdout:2363 write 0x878ddb0 thru 0x8795175 (0x73c6 bytes) 2024-09-21T22:50:39.165 INFO:teuthology.orchestra.run.smithi094.stdout:2364 writesame 0xe2d0beb thru 0xe2d6c21 data_size 0x1(0x6037 bytes) 2024-09-21T22:50:39.187 INFO:teuthology.orchestra.run.smithi094.stdout:2365 read 0x8e2db6e thru 0x8e37814 (0x9ca7 bytes) 2024-09-21T22:50:39.197 INFO:teuthology.orchestra.run.smithi094.stdout:2366 trunc from 0xe837e30 to 0x80371fd 2024-09-21T22:50:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.223 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:39.226 INFO:teuthology.orchestra.run.smithi094.stdout:2369 writesame 0x736303f thru 0x736a9fb data_size 0x17(0x79bd bytes) 2024-09-21T22:50:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:2370 write 0xafdd383 thru 0xafe0a60 (0x36de bytes) 2024-09-21T22:50:39.252 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:39.244+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:39.259 INFO:teuthology.orchestra.run.smithi094.stdout:2371 write 0x9b3e12f thru 0x9b499db (0xb8ad bytes) 2024-09-21T22:50:39.269 INFO:teuthology.orchestra.run.smithi094.stdout:2372 read 0x3793752 thru 0x37937aa (0x59 bytes) 2024-09-21T22:50:39.288 INFO:teuthology.orchestra.run.smithi094.stdout:2373 trunc from 0xafe0a61 to 0x6eeb72 2024-09-21T22:50:39.353 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:39.348+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:39.367 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.367 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.378 INFO:teuthology.orchestra.run.smithi094.stdout:2374 write 0x9364052 thru 0x9371f0a (0xdeb9 bytes) 2024-09-21T22:50:39.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:39.400 INFO:teuthology.orchestra.run.smithi094.stdout:2375 writesame 0xb1ccfa0 thru 0xb1cf4fb data_size 0x2(0x255c bytes) 2024-09-21T22:50:39.401 INFO:teuthology.orchestra.run.smithi094.stdout:2378 read 0x86fa9a thru 0x87d1a0 (0xd707 bytes) 2024-09-21T22:50:39.401 INFO:teuthology.orchestra.run.smithi094.stdout:2379 write 0x29055c8 thru 0x290829e (0x2cd7 bytes) 2024-09-21T22:50:39.409 INFO:teuthology.orchestra.run.smithi094.stdout:2380 trunc from 0xb1cf4fc to 0xa3eba86 2024-09-21T22:50:39.428 INFO:teuthology.orchestra.run.smithi094.stdout:2381 write 0xb51bbda thru 0xb528a8f (0xceb6 bytes) 2024-09-21T22:50:39.437 INFO:teuthology.orchestra.run.smithi094.stdout:2383 writesame 0xa8863cb thru 0xa88e8f0 data_size 0xd(0x8526 bytes) 2024-09-21T22:50:39.445 INFO:teuthology.orchestra.run.smithi094.stdout:2384 punch from 0x2f29944 to 0x2f34d0f, (0xb3cb bytes) 2024-09-21T22:50:39.445 INFO:teuthology.orchestra.run.smithi094.stdout:2385 read 0x3e331f7 thru 0x3e3b776 (0x8580 bytes) 2024-09-21T22:50:39.445 INFO:teuthology.orchestra.run.smithi094.stdout:2386 read 0x9b90287 thru 0x9b9fd09 (0xfa83 bytes) 2024-09-21T22:50:39.445 INFO:teuthology.orchestra.run.smithi094.stdout:2387 write 0xb2942fa thru 0xb2a2936 (0xe63d bytes) 2024-09-21T22:50:39.451 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.451 INFO:teuthology.orchestra.run.smithi094.stdout:2392 read 0x873a6be thru 0x87451cc (0xab0f bytes) 2024-09-21T22:50:39.454 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:39.448+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:39.455 INFO:teuthology.orchestra.run.smithi094.stdout:2395 trunc from 0xb528a90 to 0x89e12be 2024-09-21T22:50:39.474 INFO:teuthology.orchestra.run.smithi094.stdout:2396 punch from 0x885f9f5 to 0x886a479, (0xaa84 bytes) 2024-09-21T22:50:39.474 INFO:teuthology.orchestra.run.smithi094.stdout:2397 writesame 0x19d7587 thru 0x19db6b3 data_size 0x5(0x412d bytes) 2024-09-21T22:50:39.481 INFO:teuthology.orchestra.run.smithi094.stdout:2398 write 0x2665ad8 thru 0x266d1c0 (0x76e9 bytes) 2024-09-21T22:50:39.489 INFO:teuthology.orchestra.run.smithi094.stdout:2399 write 0x44ddc44 thru 0x44e20e0 (0x449d bytes) 2024-09-21T22:50:39.496 INFO:teuthology.orchestra.run.smithi094.stdout:2400 read 0x37a1c62 thru 0x37a4573 (0x2912 bytes) 2024-09-21T22:50:39.496 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.497 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.497 INFO:teuthology.orchestra.run.smithi094.stdout:2403 write 0x1fa4409 thru 0x1fa6a49 (0x2641 bytes) 2024-09-21T22:50:39.502 INFO:teuthology.orchestra.run.smithi094.stdout:2404 read 0x174d71b thru 0x174f0ec (0x19d2 bytes) 2024-09-21T22:50:39.502 INFO:teuthology.orchestra.run.smithi094.stdout:2405 punch from 0x2dc494a to 0x2dd37e8, (0xee9e bytes) 2024-09-21T22:50:39.515 INFO:teuthology.orchestra.run.smithi094.stdout:2406 write 0xc2eb70a thru 0xc2ed128 (0x1a1f bytes) 2024-09-21T22:50:39.533 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.534 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.535 INFO:teuthology.orchestra.run.smithi094.stdout:2407 trunc from 0xc2ed129 to 0x2cd5dbe 2024-09-21T22:50:39.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:39.553 INFO:teuthology.orchestra.run.smithi094.stdout:2408 trunc from 0x2cd5dbe to 0x11a9ed9 2024-09-21T22:50:39.555 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:39.553+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:39.571 INFO:teuthology.orchestra.run.smithi094.stdout:2409 read 0x2aa1d2 thru 0x2aaaf1 (0x920 bytes) 2024-09-21T22:50:39.571 INFO:teuthology.orchestra.run.smithi094.stdout:2410 read 0x613245 thru 0x61710e (0x3eca bytes) 2024-09-21T22:50:39.587 INFO:teuthology.orchestra.run.smithi094.stdout:2411 write 0x54ffa55 thru 0x550ea68 (0xf014 bytes) 2024-09-21T22:50:39.613 INFO:teuthology.orchestra.run.smithi094.stdout:2413 writesame 0xcbbc35f thru 0xcbc0b1f data_size 0x27(0x47c1 bytes) 2024-09-21T22:50:39.616 INFO:teuthology.orchestra.run.smithi094.stdout:2415 trunc from 0xcbc0b20 to 0xe5f7687 2024-09-21T22:50:39.625 INFO:teuthology.orchestra.run.smithi094.stdout:2416 read 0x7391997 thru 0x739bc07 (0xa271 bytes) 2024-09-21T22:50:39.625 INFO:teuthology.orchestra.run.smithi094.stdout:2417 write 0xa0a98fa thru 0xa0b76a3 (0xddaa bytes) 2024-09-21T22:50:39.635 INFO:teuthology.orchestra.run.smithi094.stdout:2418 write 0x7ac39b7 thru 0x7ac7319 (0x3963 bytes) 2024-09-21T22:50:39.640 INFO:teuthology.orchestra.run.smithi094.stdout:2419 write 0xa8df943 thru 0xa8eef6e (0xf62c bytes) 2024-09-21T22:50:39.647 INFO:teuthology.orchestra.run.smithi094.stdout:2421 write 0xaaf0b3d thru 0xaaf7fc2 (0x7486 bytes) 2024-09-21T22:50:39.651 INFO:teuthology.orchestra.run.smithi094.stdout:2422 punch from 0xaf1bb63 to 0xaf1db9a, (0x2037 bytes) 2024-09-21T22:50:39.651 INFO:teuthology.orchestra.run.smithi094.stdout:2423 punch from 0x57e1056 to 0x57ea01e, (0x8fc8 bytes) 2024-09-21T22:50:39.651 INFO:teuthology.orchestra.run.smithi094.stdout:2424 read 0x9f3e97f thru 0x9f4627c (0x78fe bytes) 2024-09-21T22:50:39.651 INFO:teuthology.orchestra.run.smithi094.stdout:2425 write 0xd8fee16 thru 0xd9047fd (0x59e8 bytes) 2024-09-21T22:50:39.656 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:39.653+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:39.663 INFO:teuthology.orchestra.run.smithi094.stdout:2426 trunc from 0xe5f7687 to 0xc14d73a 2024-09-21T22:50:39.681 INFO:teuthology.orchestra.run.smithi094.stdout:2427 punch from 0x80b8f37 to 0x80ba938, (0x1a01 bytes) 2024-09-21T22:50:39.681 INFO:teuthology.orchestra.run.smithi094.stdout:2428 writesame 0x6ded4aa thru 0x6df1f68 data_size 0x2b(0x4abf bytes) 2024-09-21T22:50:39.687 INFO:teuthology.orchestra.run.smithi094.stdout:2429 read 0x372748d thru 0x37294f8 (0x206c bytes) 2024-09-21T22:50:39.687 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.688 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.698 INFO:teuthology.orchestra.run.smithi094.stdout:2431 write 0xc5c3c7a thru 0xc5c798c (0x3d13 bytes) 2024-09-21T22:50:39.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:39.704 INFO:teuthology.orchestra.run.smithi094.stdout:2434 write 0x23c4d4d thru 0x23cdc4f (0x8f03 bytes) 2024-09-21T22:50:39.711 INFO:teuthology.orchestra.run.smithi094.stdout:2435 punch from 0x5feaf96 to 0x5ff586c, (0xa8d6 bytes) 2024-09-21T22:50:39.733 INFO:teuthology.orchestra.run.smithi094.stdout:2437 trunc from 0xc5c798d to 0x1467f0 2024-09-21T22:50:39.757 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:39.752+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:39.821 INFO:teuthology.orchestra.run.smithi094.stdout:2440 writesame 0x86db5bf thru 0x86dbc52 data_size 0x2(0x694 bytes) 2024-09-21T22:50:39.821 INFO:teuthology.orchestra.run.smithi094.stdout:2441 punch from 0x3590251 to 0x3597271, (0x7020 bytes) 2024-09-21T22:50:39.844 INFO:teuthology.orchestra.run.smithi094.stdout:2442 writesame 0xe918318 thru 0xe923d93 data_size 0x9b(0xba7c bytes) 2024-09-21T22:50:39.844 INFO:teuthology.orchestra.run.smithi094.stdout:2445 punch from 0x509d2c to 0x514e1b, (0xb0ef bytes) 2024-09-21T22:50:39.844 INFO:teuthology.orchestra.run.smithi094.stdout:2447 write 0x8f4b95 thru 0x9027e4 (0xdc50 bytes) 2024-09-21T22:50:39.852 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:39.852 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:39.854 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.854 INFO:teuthology.orchestra.run.smithi094.stdout:2450 punch from 0xd48f559 to 0xd49a97c, (0xb423 bytes) 2024-09-21T22:50:39.858 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:39.852+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:39.863 INFO:teuthology.orchestra.run.smithi094.stdout:2451 trunc from 0xe923d94 to 0x964e5b0 2024-09-21T22:50:39.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:39.875 INFO:teuthology.orchestra.run.smithi094.stdout:2454 trunc from 0x964e5b0 to 0xaa6d00c 2024-09-21T22:50:39.887 INFO:teuthology.orchestra.run.smithi094.stdout:2456 trunc from 0xaa6d00c to 0xc38e37a 2024-09-21T22:50:39.895 INFO:teuthology.orchestra.run.smithi094.stdout:2457 read 0xac90959 thru 0xac9e3b5 (0xda5d bytes) 2024-09-21T22:50:39.895 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.896 INFO:teuthology.orchestra.run.smithi094.stdout:2459 read 0x99f18dc thru 0x99f3e0f (0x2534 bytes) 2024-09-21T22:50:39.896 INFO:teuthology.orchestra.run.smithi094.stdout:2460 writesame 0x25faa17 thru 0x2606c01 data_size 0x1(0xc1eb bytes) 2024-09-21T22:50:39.921 INFO:teuthology.orchestra.run.smithi094.stdout:2463 read 0x3f39cea thru 0x3f4200e (0x8325 bytes) 2024-09-21T22:50:39.921 INFO:teuthology.orchestra.run.smithi094.stdout:2464 writesame 0xb66a8c4 thru 0xb66ffde data_size 0x1(0x571b bytes) 2024-09-21T22:50:39.948 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:39.949 INFO:teuthology.orchestra.run.smithi094.stdout:2467 punch from 0x8ff1db8 to 0x8ffb153, (0x939b bytes) 2024-09-21T22:50:39.949 INFO:teuthology.orchestra.run.smithi094.stdout:2468 read 0xbc1e560 thru 0xbc1f334 (0xdd5 bytes) 2024-09-21T22:50:39.949 INFO:teuthology.orchestra.run.smithi094.stdout:2469 punch from 0x230d436 to 0x2311180, (0x3d4a bytes) 2024-09-21T22:50:39.954 INFO:teuthology.orchestra.run.smithi094.stdout:2470 trunc from 0xc38e37a to 0x966458b 2024-09-21T22:50:39.958 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:39.957+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:39.971 INFO:teuthology.orchestra.run.smithi094.stdout:2472 read 0x2b5a532 thru 0x2b68891 (0xe360 bytes) 2024-09-21T22:50:39.971 INFO:teuthology.orchestra.run.smithi094.stdout:2474 read 0x588de94 thru 0x5893573 (0x56e0 bytes) 2024-09-21T22:50:39.988 INFO:teuthology.orchestra.run.smithi094.stdout:2478 write 0xc788900 thru 0xc796995 (0xe096 bytes) 2024-09-21T22:50:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:2479 trunc from 0xc796996 to 0xaccdfe8 2024-09-21T22:50:40.005 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:40.005 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:40.019 INFO:teuthology.orchestra.run.smithi094.stdout:2480 write 0x820d5a thru 0x82d974 (0xcc1b bytes) 2024-09-21T22:50:40.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:40.040 INFO:teuthology.orchestra.run.smithi094.stdout:2481 trunc from 0xaccdfe8 to 0x2907f6e 2024-09-21T22:50:40.067 INFO:teuthology.orchestra.run.smithi094.stdout:2484 trunc from 0x2907f6e to 0xda46524 2024-09-21T22:50:40.068 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:40.060+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:40.075 INFO:teuthology.orchestra.run.smithi094.stdout:2486 writesame 0xa20b9cc thru 0xa21167f data_size 0x22(0x5cb4 bytes) 2024-09-21T22:50:40.083 INFO:teuthology.orchestra.run.smithi094.stdout:2487 punch from 0xa7f584c to 0xa7fa4ed, (0x4ca1 bytes) 2024-09-21T22:50:40.084 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:40.084 INFO:teuthology.orchestra.run.smithi094.stdout:2489 write 0x2dc271e thru 0x2dc95a4 (0x6e87 bytes) 2024-09-21T22:50:40.092 INFO:teuthology.orchestra.run.smithi094.stdout:2490 write 0xb16d542 thru 0xb177655 (0xa114 bytes) 2024-09-21T22:50:40.103 INFO:teuthology.orchestra.run.smithi094.stdout:2491 trunc from 0xda46524 to 0xaf36258 2024-09-21T22:50:40.132 INFO:teuthology.orchestra.run.smithi094.stdout:2492 writesame 0xe0ed0da thru 0xe0fb8f9 data_size 0x30(0xe820 bytes) 2024-09-21T22:50:40.132 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:40.133 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:40.133 INFO:teuthology.orchestra.run.smithi094.stdout:2495 read 0xcc96dd8 thru 0xcc98404 (0x162d bytes) 2024-09-21T22:50:40.133 INFO:teuthology.orchestra.run.smithi094.stdout:2496 write 0x5825215 thru 0x5828f1b (0x3d07 bytes) 2024-09-21T22:50:40.138 INFO:teuthology.orchestra.run.smithi094.stdout:2497 write 0x7e07ec8 thru 0x7e10e4c (0x8f85 bytes) 2024-09-21T22:50:40.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:40.147 INFO:teuthology.orchestra.run.smithi094.stdout:2498 trunc from 0xe0fb8fa to 0xe4300af 2024-09-21T22:50:40.155 INFO:teuthology.orchestra.run.smithi094.stdout:2500 writesame 0x1f48d5d thru 0x1f54576 data_size 0xa(0xb81a bytes) 2024-09-21T22:50:40.160 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:40.152+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:40.165 INFO:teuthology.orchestra.run.smithi094.stdout:2501 write 0x53d768f thru 0x53e2024 (0xa996 bytes) 2024-09-21T22:50:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:2502 clone 12 order 18 su 2048 sc 7 2024-09-21T22:50:40.178 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:40.178 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:40.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:40.261 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:40.257+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:40.285 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:40.302 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:40.303 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:40.319 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:40.319 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:40.337 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:40.337 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:40.356 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:40.356 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:40.361 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:40.356+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:40.368 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:40.368 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:40.377 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:40.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:40.398 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:40.398 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:40.419 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:40.463 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:40.461+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:40.516 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:40.516 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:40.516 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:40.516 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:40.516 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:40.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:40.563 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:40.561+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:620 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:40.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.980328361000002, 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014257994, 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 620 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673476+0000", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673479+0000", 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5560000000000002e-06 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673495+0000", 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.615e-05 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673500+0000", 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9439999999999996e-06 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684881+0000", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011381242 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.684990+0000", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109197 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.685289+0000", 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029849100000000001 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687532+0000", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022436510000000002 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687588+0000", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5655999999999997e-05 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687652+0000", 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3544e-05 2024-09-21T22:50:40.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687734+0000", 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2126999999999995e-05 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:621 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.980297697000001, 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018613351, 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 621 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673506+0000", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673507+0000", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673510+0000", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8059999999999999e-06 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673516+0000", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3099999999999997e-06 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.673520+0000", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4619999999999998e-06 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686563+0000", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013042876 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.686663+0000", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9570999999999999e-05 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687059+0000", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039618800000000001 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.687883+0000", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082348899999999995 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692061+0000", 2024-09-21T22:50:40.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004178022 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692094+0000", 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3473999999999998e-05 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.692120+0000", 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5711000000000002e-05 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:680 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.907621334000002, 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013778161000000001, 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 680 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746183+0000", 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746187+0000", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9160000000000003e-06 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746193+0000", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.746200+0000", 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5760000000000002e-06 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753332+0000", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071319840000000001 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753409+0000", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7111000000000001e-05 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.753698+0000", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028885599999999999 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.754451+0000", 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075351400000000005 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759904+0000", 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054529679999999999 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759937+0000", 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3018999999999997e-05 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:12.759961+0000", 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3889000000000001e-05 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1046 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470501682999998, 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015794288, 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1046 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183302+0000", 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183304+0000", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1930000000000001e-06 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183311+0000", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3339999999999996e-06 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183339+0000", 2024-09-21T22:50:40.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8030000000000001e-05 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183346+0000", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.192825+0000", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094795499999999998 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193017+0000", 2024-09-21T22:50:40.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000192287 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.193797+0000", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000779388 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197391+0000", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035944969999999999 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198914+0000", 2024-09-21T22:50:40.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015223039999999999 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199027+0000", 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113127 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199097+0000", 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0024999999999994e-05 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1047 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470449058, 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017348148000000001, 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1047 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183355+0000", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183356+0000", 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183362+0000", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.806e-06 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183371+0000", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5529999999999998e-06 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183378+0000", 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8689999999999998e-06 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.194941+0000", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011562767999999999 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195056+0000", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115281 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195493+0000", 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043665799999999998 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199120+0000", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036272919999999998 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200347+0000", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001226782 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200566+0000", 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021947500000000001 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200703+0000", 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013704200000000001 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1048 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470416733, 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017543792999999998, 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1048 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183387+0000", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183388+0000", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183392+0000", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7139999999999999e-06 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183401+0000", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2469999999999997e-06 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183408+0000", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0469999999999996e-06 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.195923+0000", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012514658 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196021+0000", 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8113000000000001e-05 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.196394+0000", 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037272199999999997 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200738+0000", 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043443179999999998 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200803+0000", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5166000000000006e-05 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200869+0000", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6190000000000002e-05 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.200931+0000", 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1996999999999996e-05 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1049 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470387098, 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017955736999999999, 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1049 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.692 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183417+0000", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183418+0000", 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:40.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183421+0000", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5580000000000001e-06 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183431+0000", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3430000000000002e-06 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183438+0000", 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-21T22:50:40.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197463+0000", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014025892999999999 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.197596+0000", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013223600000000001 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.198242+0000", 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064663999999999998 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201081+0000", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002839144 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201120+0000", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8563000000000003e-05 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201262+0000", 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014153000000000001 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201373+0000", 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111236 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1050 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470357237999998, 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020333185, 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1050 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183447+0000", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183448+0000", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183451+0000", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6389999999999998e-06 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183460+0000", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2569999999999996e-06 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183467+0000", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9600000000000003e-06 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199154+0000", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015686734000000001 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199259+0000", 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105364 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.199748+0000", 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048820500000000002 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203483+0000", 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003735787 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203519+0000", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5954000000000002e-05 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203653+0000", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013328799999999999 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203780+0000", 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012739399999999999 2024-09-21T22:50:40.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1051 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470303856000001, 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021843896000000002, 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1051 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183500+0000", 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183501+0000", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2399999999999997e-07 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183506+0000", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183527+0000", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1353e-05 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183535+0000", 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0180000000000001e-06 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201414+0000", 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01787888 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201496+0000", 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2201999999999997e-05 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.201824+0000", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032739100000000001 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205098+0000", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032737280000000001 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205130+0000", 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2805000000000002e-05 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205258+0000", 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000127911 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205344+0000", 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5882000000000002e-05 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1052 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470256030000002, 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022039640999999999, 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1052 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183548+0000", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183549+0000", 2024-09-21T22:50:40.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183553+0000", 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1130000000000003e-06 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183563+0000", 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6930000000000002e-06 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183570+0000", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202302+0000", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018731863000000001 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202413+0000", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111117 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.202922+0000", 2024-09-21T22:50:40.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050897300000000002 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205382+0000", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024604240000000001 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205403+0000", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1058999999999999e-05 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205503+0000", 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100068 2024-09-21T22:50:40.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205588+0000", 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4584000000000002e-05 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1053 3.2 3:424b4911:::rbd_object_map.111ecbf88771:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.470224940000001, 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.022880587000000001, 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1053 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183579+0000", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183580+0000", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183592+0000", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.224e-05 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183602+0000", 2024-09-21T22:50:40.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094e-05 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.183609+0000", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1810000000000004e-06 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203845+0000", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020235907000000001 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.203896+0000", 2024-09-21T22:50:40.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1187999999999998e-05 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204112+0000", 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021558999999999999 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205609+0000", 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001496743 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206370+0000", 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076130800000000004 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206421+0000", 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0491000000000002e-05 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.206460+0000", 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9218999999999998e-05 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1054 3.2 3:46641ad1:::rbd_data.111ecbf88771.0000000000000000:head [stat,truncate 204800] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.466028129000001, 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019736885999999999, 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1054 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187776+0000", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187777+0000", 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187783+0000", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2780000000000001e-06 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187794+0000", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1464999999999999e-05 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.187811+0000", 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6716999999999999e-05 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204451+0000", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016639856000000001 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204504+0000", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3763e-05 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.204604+0000", 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9433999999999997e-05 2024-09-21T22:50:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.205621+0000", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010175049999999999 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207340+0000", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718884 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207409+0000", 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8959000000000002e-05 2024-09-21T22:50:40.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:13.207513+0000", 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010366600000000001 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:1805 3.b 3:d5e4ffc8:::rbd_data.111ea6ec99f1.000000000000009d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.595062942999999, 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.032681931999999997, 2024-09-21T22:50:40.725 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1805 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058741+0000", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058743+0000", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.37e-06 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058751+0000", 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0760000000000003e-06 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058769+0000", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7927e-05 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058780+0000", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1353e-05 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.058934+0000", 2024-09-21T22:50:40.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015403899999999999 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.059298+0000", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036453700000000001 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.065146+0000", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058479789999999997 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091319+0000", 2024-09-21T22:50:40.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026172104000000002 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091334+0000", 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4939999999999999e-05 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091379+0000", 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5593000000000002e-05 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:14.091423+0000", 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4014000000000002e-05 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3479 3.7 3:e3964c56:::rbd_data.111e7572a6e.0000000000000000:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.460181744, 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01465428, 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:40.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3479 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193622+0000", 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193624+0000", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9199999999999998e-06 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193634+0000", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0132999999999999e-05 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193666+0000", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1405999999999997e-05 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193684+0000", 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8185999999999998e-05 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193764+0000", 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0051999999999999e-05 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.193901+0000", 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013736 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.208277+0000", 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014375222999999999 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:3487 3.b 3:d69a8a6f:::rbd_data.111e7572a6e.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.459241088999999, 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01338204, 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:40.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3487 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194563+0000", 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194564+0000", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4470000000000001e-06 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194573+0000", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0630000000000006e-06 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194596+0000", 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3312999999999999e-05 2024-09-21T22:50:40.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194611+0000", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194681+0000", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9865999999999998e-05 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.194803+0000", 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012176900000000001 2024-09-21T22:50:40.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.207945+0000", 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013141972 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6462 3.2 3:44ccee00:::rbd_data.111ebc806776.000000000000001a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.891631954000001, 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.08074365, 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6462 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:40.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762174+0000", 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3029999999999998e-06 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762186+0000", 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.136e-05 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762217+0000", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0663000000000001e-05 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762232+0000", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5099999999999999e-05 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762388+0000", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000156362 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762632+0000", 2024-09-21T22:50:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243995 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.777316+0000", 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014684262999999999 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842690+0000", 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065373938000000006 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842722+0000", 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2283999999999998e-05 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842825+0000", 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102192 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842916+0000", 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1189999999999999e-05 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:6462 3.2 e34/24 ondisk, result = 0)", 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.832218052000002, 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021121876000000001, 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:40.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821586+0000", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821588+0000", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0600000000000002e-06 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821598+0000", 2024-09-21T22:50:40.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8160000000000005e-06 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821633+0000", 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4975999999999999e-05 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.821651+0000", 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8287e-05 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842659+0000", 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.021007426999999999 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842677+0000", 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8533e-05 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842708+0000", 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0777e-05 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:9376 3.2 3:4315e470:::rbd_data.111e33663d62.0000000000000004:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.236908931, 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013487305, 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 9376 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416895+0000", 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416896+0000", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8500000000000001e-07 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416900+0000", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7450000000000001e-06 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416907+0000", 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7360000000000001e-06 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.416913+0000", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.418988+0000", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020751480000000002 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419086+0000", 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7659999999999997e-05 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.419282+0000", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019622899999999999 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.420737+0000", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455582 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430236+0000", 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094984900000000001 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430316+0000", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0382000000000004e-05 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:28.430383+0000", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6178000000000002e-05 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:11367 3.1e 3:7cc77dab:::rbd_data.111ea12b32db.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8393346210000008, 2024-09-21T22:50:40.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044095079000000002, 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11367 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814470+0000", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814472+0000", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0930000000000001e-06 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814483+0000", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1293e-05 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814512+0000", 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9373000000000002e-05 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814527+0000", 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4589999999999999e-05 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814641+0000", 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114339 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.814787+0000", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014541900000000001 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858463+0000", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043676404000000002 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858508+0000", 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4731000000000003e-05 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:31.858565+0000", 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6836999999999998e-05 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13534 3.1e 3:7f270d40:::rbd_data.111e4c6c7d62.000000000000022a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.6271469789999999, 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094102410000001, 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:40.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13534 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:40.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026659+0000", 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.544e-06 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026666+0000", 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0679999999999999e-06 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:40.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026681+0000", 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4758e-05 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026720+0000", 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9595999999999998e-05 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026822+0000", 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101512 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026833+0000", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1698999999999999e-05 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.031773+0000", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049400070000001 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032053+0000", 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027928499999999999 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032241+0000", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000188448 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.034179+0000", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001938214 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035722+0000", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001542762 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035951+0000", 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000229232 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.036067+0000", 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116116 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:40.765 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:40.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:40.767 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:40.764+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:40.868 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:40.865+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:40.897 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:40.897 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:40.907 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:50:40.907 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:40.907 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T22:50:40.907 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T22:50:40.907 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T22:50:40.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T22:50:40.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:40.997 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:40.997+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:41.044 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:41.044 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T22:50:41.044 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:41.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:50:41.069 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:41.064+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:41.083 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:41.084 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:41.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:41.124 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone11 from 0xe4300af (overlap 0x1467f0) to 0x61f2e 2024-09-21T22:50:41.169 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:41.164+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:41.217 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T22:50:41.273 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:41.264+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:11225 3.1d e40/24 3:bfcb9502:::rbd_data.111ea12b32db.0000000000000000:head v 40'541, mlcod=38'539)", 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.7615602129999992, 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020663840999999999, 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514416+0000", 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.514418+0000", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2960000000000001e-06 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518906+0000", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044876899999999999 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518949+0000", 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2960000000000002e-05 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.518968+0000", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9395999999999999e-05 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.519165+0000", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019721300000000001 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.519210+0000", 2024-09-21T22:50:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4419999999999998e-05 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535001+0000", 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015791052 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535080+0000", 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8813999999999995e-05 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11629 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4385966490000008, 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017211015999999999, 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11629 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837379+0000", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837380+0000", 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8599999999999998e-07 2024-09-21T22:50:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837384+0000", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2629999999999997e-06 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837394+0000", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8300000000000008e-06 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837405+0000", 2024-09-21T22:50:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0779e-05 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.844652+0000", 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072473060000000002 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.844915+0000", 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026219600000000002 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.846043+0000", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001128275 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.849721+0000", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036786119999999999 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854498+0000", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047762430000000003 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854551+0000", 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3158999999999997e-05 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854590+0000", 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666999999999998e-05 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11630 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4385612119999998, 2024-09-21T22:50:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018432139, 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11630 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837415+0000", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837416+0000", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837420+0000", 2024-09-21T22:50:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2259999999999999e-06 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837429+0000", 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4460000000000007e-06 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837437+0000", 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4699999999999996e-06 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.847532+0000", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010094893000000001 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.847688+0000", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000156603 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.848375+0000", 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068659400000000001 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852052+0000", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036775219999999999 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855725+0000", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036722859999999999 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855797+0000", 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2878000000000003e-05 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855847+0000", 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9570999999999997e-05 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11631 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4385298909999999, 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018496809999999999, 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11631 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837446+0000", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837447+0000", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837451+0000", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9029999999999997e-06 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837460+0000", 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2960000000000004e-06 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837467+0000", 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2169999999999997e-06 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.849884+0000", 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012417167999999999 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.850109+0000", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000224582 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.851056+0000", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094672699999999996 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854725+0000", 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036696279999999999 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855857+0000", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011317829999999999 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855898+0000", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1182e-05 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855943+0000", 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4684000000000001e-05 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11632 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4384995529999998, 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020031846999999998, 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11632 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837477+0000", 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3e-07 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837481+0000", 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4570000000000001e-06 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837493+0000", 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2702e-05 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837501+0000", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852143+0000", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014642654 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.852358+0000", 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000214985 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.853448+0000", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010893680000000001 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856050+0000", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026024759999999998 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857364+0000", 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013136210000000001 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857442+0000", 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8516999999999994e-05 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857508+0000", 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6230999999999997e-05 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11633 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4384660619999998, 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020634830999999999, 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11633 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837510+0000", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837511+0000", 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.44e-07 2024-09-21T22:50:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837515+0000", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8299999999999998e-06 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837524+0000", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1889999999999996e-06 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837531+0000", 2024-09-21T22:50:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1760000000000004e-06 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854772+0000", 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017241480999999999 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.854844+0000", 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1320000000000002e-05 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.855136+0000", 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029208499999999998 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857531+0000", 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023956279999999999 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.857848+0000", 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031616400000000002 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858024+0000", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017639500000000001 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858145+0000", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120919 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11634 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4384359700000005, 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023090243, 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11634 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837540+0000", 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837541+0000", 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1999999999999999e-07 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837545+0000", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0500000000000002e-06 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837554+0000", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2450000000000004e-06 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.837561+0000", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3830000000000003e-06 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856110+0000", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018548565999999999 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856220+0000", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110226 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.856770+0000", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055009000000000004 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.858496+0000", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001725578 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860323+0000", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018268080000000001 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860498+0000", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175582 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.860630+0000", 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013209499999999999 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11650 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4100870630000006, 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019674952999999998, 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11650 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865889+0000", 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9400000000000001e-07 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865892+0000", 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.802e-06 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865898+0000", 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5079999999999996e-06 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865902+0000", 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6779999999999999e-06 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.876555+0000", 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010652134000000001 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.876799+0000", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024472799999999999 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.877868+0000", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010684049999999999 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881258+0000", 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033901510000000001 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885471+0000", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042133170000000003 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885520+0000", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8467999999999997e-05 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885564+0000", 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4367999999999999e-05 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11651 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4100675430000003, 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019787837999999999, 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11651 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865909+0000", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.65e-07 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865912+0000", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6350000000000002e-06 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865917+0000", 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3369999999999999e-06 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865921+0000", 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.879328+0000", 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013407117 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.879489+0000", 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160278 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.880085+0000", 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059590499999999998 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881376+0000", 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001291798 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885586+0000", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00420959 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885657+0000", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1483999999999995e-05 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885696+0000", 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9053e-05 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11652 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4100489809999992, 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019872035999999999, 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11652 2024-09-21T22:50:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865927+0000", 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865928+0000", 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7800000000000002e-07 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865930+0000", 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5019999999999999e-06 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865935+0000", 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3179999999999997e-06 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865940+0000", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1540000000000004e-06 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881478+0000", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015538847999999999 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.881623+0000", 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014447 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882055+0000", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043187100000000001 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885404+0000", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033497190000000001 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885703+0000", 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029899599999999999 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885764+0000", 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0492000000000003e-05 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885799+0000", 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5287999999999998e-05 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11653 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4100271870000007, 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019953525999999999, 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11653 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865949+0000", 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3599999999999999e-07 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865952+0000", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6850000000000001e-06 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865958+0000", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.857e-06 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865962+0000", 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3490000000000002e-06 2024-09-21T22:50:41.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882603+0000", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016640818000000002 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.882697+0000", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4203000000000006e-05 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.883711+0000", 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001013835 2024-09-21T22:50:41.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885460+0000", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017489579999999999 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885806+0000", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000346168 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885849+0000", 2024-09-21T22:50:41.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.295e-05 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885902+0000", 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3266999999999997e-05 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11654 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4100079169999997, 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021916723999999999, 2024-09-21T22:50:41.341 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11654 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865968+0000", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865969+0000", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.65e-07 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865971+0000", 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6350000000000002e-06 2024-09-21T22:50:41.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865976+0000", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1970000000000004e-06 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.865981+0000", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5480000000000003e-06 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884316+0000", 2024-09-21T22:50:41.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018334613 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884425+0000", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010932 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.884873+0000", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00044802900000000002 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.885970+0000", 2024-09-21T22:50:41.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010975410000000001 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887655+0000", 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001685001 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887806+0000", 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150536 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:32.887885+0000", 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8938999999999998e-05 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11812 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.2608126760000005, 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015836000999999999, 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11812 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015163+0000", 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015164+0000", 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.08e-07 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015167+0000", 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.993e-06 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015173+0000", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5529999999999998e-06 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.015179+0000", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1490000000000002e-06 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.028702+0000", 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013523599000000001 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.028783+0000", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0817999999999998e-05 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.029222+0000", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043925399999999998 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030263+0000", 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010406809999999999 2024-09-21T22:50:41.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030944+0000", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068143100000000003 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030976+0000", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1547000000000001e-05 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.030999+0000", 2024-09-21T22:50:41.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3567999999999999e-05 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13363 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.4058343710000001, 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016584944000000001, 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13363 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870142+0000", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000001e-07 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870145+0000", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870153+0000", 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-21T22:50:41.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870158+0000", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.045e-06 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.882999+0000", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012840356000000001 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.883090+0000", 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1393999999999999e-05 2024-09-21T22:50:41.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.883490+0000", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039968000000000002 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885780+0000", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002290566 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886506+0000", 2024-09-21T22:50:41.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072612999999999996 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886622+0000", 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115968 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886727+0000", 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104251 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13364 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.4058099100000003, 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016765598, 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13364 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.359 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:41.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870166+0000", 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870167+0000", 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6100000000000001e-07 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870179+0000", 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2513e-05 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870187+0000", 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7789999999999996e-06 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.870192+0000", 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3870000000000003e-06 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884122+0000", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013930120000000001 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884207+0000", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4359999999999999e-05 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.884627+0000", 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042007699999999998 2024-09-21T22:50:41.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.885953+0000", 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013261049999999999 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886750+0000", 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00079673700000000005 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886835+0000", 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5037000000000006e-05 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:35.886932+0000", 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7021999999999994e-05 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13619 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T22:50:41.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:37.149240+0000", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.126735751, 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015913255000000001, 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13619 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149240+0000", 2024-09-21T22:50:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149240+0000", 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149241+0000", 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149245+0000", 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5370000000000002e-06 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149253+0000", 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9020000000000005e-06 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149260+0000", 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.158143+0000", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088831410000000006 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.158358+0000", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021440000000000001 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.159068+0000", 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00071068300000000004 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.164869+0000", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058010789999999998 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.164952+0000", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2447999999999994e-05 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165018+0000", 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6439000000000005e-05 2024-09-21T22:50:41.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165154+0000", 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013526200000000001 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13620 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:37.149269+0000", 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.126706853, 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016414755999999999, 2024-09-21T22:50:41.372 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13620 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149269+0000", 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149269+0000", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149270+0000", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6000000000000003e-07 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149274+0000", 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6040000000000001e-06 2024-09-21T22:50:41.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149282+0000", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7930000000000003e-06 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149289+0000", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8120000000000002e-06 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.159834+0000", 2024-09-21T22:50:41.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010544714 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.159945+0000", 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111389 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.160602+0000", 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00065698199999999997 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165326+0000", 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047236910000000003 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165362+0000", 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000001e-05 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165564+0000", 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020194500000000001 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165684+0000", 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119616 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13621 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:37.149310+0000", 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1266662150000002, 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016688946, 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13621 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149310+0000", 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149310+0000", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149311+0000", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149315+0000", 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1570000000000002e-06 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149347+0000", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2320999999999998e-05 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149356+0000", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3330000000000003e-06 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.161766+0000", 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012409598000000001 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.161865+0000", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8733999999999994e-05 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.162233+0000", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036874100000000001 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165785+0000", 2024-09-21T22:50:41.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035511760000000001 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165812+0000", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7535e-05 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165924+0000", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011181899999999999 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.165999+0000", 2024-09-21T22:50:41.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4841999999999999e-05 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13622 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:37.149369+0000", 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1266075710000001, 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019350335999999999, 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13622 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149369+0000", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149369+0000", 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149369+0000", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7599999999999996e-07 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149385+0000", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5945999999999999e-05 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149396+0000", 2024-09-21T22:50:41.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.149403+0000", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.163147+0000", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013744080000000001 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.163288+0000", 2024-09-21T22:50:41.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140977 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.164082+0000", 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00079374299999999997 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.167010+0000", 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029281450000000001 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.168382+0000", 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013713830000000001 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.168606+0000", 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000224047 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.168719+0000", 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113297 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13706 3.1a 3:5b186deb:::rbd_object_map.111e4c6c7d62:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:37.247455+0000", 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0285210180000002, 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017351644999999999, 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13706 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.390 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247455+0000", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247455+0000", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247455+0000", 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0100000000000001e-07 2024-09-21T22:50:41.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247458+0000", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247463+0000", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6729999999999997e-06 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.247466+0000", 2024-09-21T22:50:41.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9689999999999999e-06 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.255832+0000", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083661120000000002 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.255950+0000", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011749299999999999 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.256330+0000", 2024-09-21T22:50:41.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038009300000000002 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.260192+0000", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00386207 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.264698+0000", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045061190000000003 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.264766+0000", 2024-09-21T22:50:41.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8245999999999997e-05 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.264807+0000", 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0432999999999997e-05 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.395 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:41.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:41.397 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:41.392+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:41.421 INFO:teuthology.orchestra.run.smithi094.stdout:2504 punch from 0x198e9d1 to 0x199bb9b, (0xd1ca bytes) 2024-09-21T22:50:41.427 INFO:teuthology.orchestra.run.smithi094.stdout:2505 writesame 0x8f6f967 thru 0x8f7cbff data_size 0x1(0xd299 bytes) 2024-09-21T22:50:41.452 INFO:teuthology.orchestra.run.smithi094.stdout:2507 trunc from 0xe4300af to 0xa466489 2024-09-21T22:50:41.497 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:41.492+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:41.536 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:41.536 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:41.536 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:41.536 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:41.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:41.598 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:41.597+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:41.671 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:41.672 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:41.672 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:41.672 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:41.672 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:41.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:41.729 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:41.729+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:41.800 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:41.792+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:41.811 INFO:teuthology.orchestra.run.smithi094.stdout:2509 read 0x350569e thru 0x35112d3 (0xbc36 bytes) 2024-09-21T22:50:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:2511 write 0x217f2cf thru 0x218d180 (0xdeb2 bytes) 2024-09-21T22:50:41.827 INFO:teuthology.orchestra.run.smithi094.stdout:2512 punch from 0x74fe4b8 to 0x75082d6, (0x9e1e bytes) 2024-09-21T22:50:41.827 INFO:teuthology.orchestra.run.smithi094.stdout:2513 write 0x63042c8 thru 0x630f4d9 (0xb212 bytes) 2024-09-21T22:50:41.840 INFO:teuthology.orchestra.run.smithi094.stdout:2514 punch from 0x5ffb1da to 0x60001ac, (0x4fd2 bytes) 2024-09-21T22:50:41.848 INFO:teuthology.orchestra.run.smithi094.stdout:2515 trunc from 0xa466489 to 0xe2fee3d 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9308 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.855 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460714324, 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030613436000000001, 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9308 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391671+0000", 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391672+0000", 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391681+0000", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3389999999999999e-06 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391700+0000", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8725000000000001e-05 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391714+0000", 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3839000000000001e-05 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.421988+0000", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03027432 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422137+0000", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014888499999999999 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422284+0000", 2024-09-21T22:50:41.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014702500000000001 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9309 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460650679, 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030903685, 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9309 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391734+0000", 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391736+0000", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2729999999999999e-06 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391741+0000", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9100000000000002e-06 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391761+0000", 2024-09-21T22:50:41.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9565e-05 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.391775+0000", 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3596e-05 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422343+0000", 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030568828999999999 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422495+0000", 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015208600000000001 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422638+0000", 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014242599999999999 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9319 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:41.864 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460346617999999, 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030911366999999999, 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9319 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392038+0000", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392040+0000", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.553e-06 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392046+0000", 2024-09-21T22:50:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1360000000000002e-06 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392065+0000", 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392079+0000", 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3815000000000001e-05 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422311+0000", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030232335999999999 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422480+0000", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000169331 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422950+0000", 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00046917699999999998 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9320 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460285526, 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031731672000000002, 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9320 2024-09-21T22:50:41.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392099+0000", 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392100+0000", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0470000000000001e-06 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392118+0000", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7184999999999999e-05 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392138+0000", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0795e-05 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392153+0000", 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4309e-05 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423024+0000", 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030871742000000001 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423635+0000", 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061065999999999996 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423831+0000", 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019593400000000001 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9321 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460211632, 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032090151999999997, 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9321 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392173+0000", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392175+0000", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-06 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392182+0000", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.746e-06 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392201+0000", 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8402000000000001e-05 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392209+0000", 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.636e-06 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423902+0000", 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031692531000000003 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424088+0000", 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018566599999999999 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424263+0000", 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017582399999999999 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.877 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9328 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 98304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460164032, 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031544833000000001, 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9328 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392221+0000", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392222+0000", 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1399999999999996e-07 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392225+0000", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5389999999999999e-06 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392235+0000", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0003000000000001e-05 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392243+0000", 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.542e-06 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422696+0000", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030453424 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.422834+0000", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137539 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423766+0000", 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093207200000000004 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9329 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 425984~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460132491, 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031920959999999998, 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9329 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392252+0000", 2024-09-21T22:50:41.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392253+0000", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392256+0000", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2729999999999998e-06 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392266+0000", 2024-09-21T22:50:41.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3700000000000001e-06 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392273+0000", 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3459999999999996e-06 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423823+0000", 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031550337999999997 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.423991+0000", 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000167932 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424173+0000", 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000182051 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9330 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.46010266, 2024-09-21T22:50:41.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032524491000000003, 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9330 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392282+0000", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392283+0000", 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392286+0000", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.864e-06 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392299+0000", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3382e-05 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392306+0000", 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2760000000000003e-06 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424267+0000", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031960323999999998 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424448+0000", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181423 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424807+0000", 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035859899999999997 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9331 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460069339, 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033079164000000001, 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9331 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392316+0000", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5400000000000001e-07 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392319+0000", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9359999999999999e-06 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392329+0000", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3829999999999998e-06 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392339+0000", 2024-09-21T22:50:41.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424890+0000", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032550548999999998 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425181+0000", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029145400000000002 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425395+0000", 2024-09-21T22:50:41.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000213573 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.460036491, 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459993061, 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:41.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459884469, 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:41.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9341 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459823826999999, 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032779214000000001, 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9341 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392561+0000", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392562+0000", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0240000000000001e-06 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392568+0000", 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3990000000000003e-06 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392585+0000", 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7125e-05 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392599+0000", 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3946e-05 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424377+0000", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031778433000000002 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.424536+0000", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159304 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425340+0000", 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080398299999999998 2024-09-21T22:50:41.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9342 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 557056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459768041, 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033794164000000002, 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9342 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392617+0000", 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392618+0000", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392623+0000", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392641+0000", 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7805999999999999e-05 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392654+0000", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3259e-05 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425439+0000", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032784424999999999 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425626+0000", 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018740799999999999 2024-09-21T22:50:41.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426411+0000", 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078503999999999998 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9343 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459712438, 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034396423000000002, 2024-09-21T22:50:41.916 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9343 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392672+0000", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392674+0000", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392679+0000", 2024-09-21T22:50:41.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6629999999999998e-06 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392697+0000", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7518999999999999e-05 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392714+0000", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668e-05 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426525+0000", 2024-09-21T22:50:41.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033811433000000002 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.426759+0000", 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023436299999999999 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427069+0000", 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000309594 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9344 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459629485000001, 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034732139000000002, 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9344 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392755+0000", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392757+0000", 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4929999999999999e-06 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392775+0000", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7733000000000001e-05 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392794+0000", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9134999999999999e-05 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392812+0000", 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8238000000000001e-05 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427147+0000", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034335093999999997 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427339+0000", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001917 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427488+0000", 2024-09-21T22:50:41.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000148746 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9345 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459552097, 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034982737999999999, 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9345 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392833+0000", 2024-09-21T22:50:41.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392834+0000", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.373e-06 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392840+0000", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9030000000000001e-06 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392861+0000", 2024-09-21T22:50:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0911999999999999e-05 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392874+0000", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2896e-05 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427564+0000", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034689953000000003 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427715+0000", 2024-09-21T22:50:41.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015096599999999999 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427816+0000", 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100735 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9346 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.459492772000001, 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380256999999998, 2024-09-21T22:50:41.929 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9346 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392893+0000", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392898+0000", 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1429999999999997e-06 2024-09-21T22:50:41.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392918+0000", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0165000000000001e-05 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392932+0000", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3073e-05 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427888+0000", 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034956787000000003 2024-09-21T22:50:41.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428089+0000", 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020047700000000001 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428272+0000", 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000183575 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.933 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.450522883, 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:41.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:41.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.937 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:11225 3.1d 3:bfcb9502:::rbd_data.111ea12b32db.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.347018264000001, 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031990183999999998, 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11225 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505367+0000", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.505370+0000", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:41.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512142+0000", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067726949999999996 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512177+0000", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5167999999999999e-05 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512196+0000", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8882e-05 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:41.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512384+0000", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018725 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.512632+0000", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024841799999999998 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.513884+0000", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001252038 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.535351+0000", 2024-09-21T22:50:41.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021467211999999999 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537246+0000", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001894864 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537305+0000", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8570999999999999e-05 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:31.537357+0000", 2024-09-21T22:50:41.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2166000000000002e-05 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:41.944 INFO:teuthology.orchestra.run.smithi094.stdout:2516 read 0xcbc7cfc thru 0xcbd0ec5 (0x91ca bytes) 2024-09-21T22:50:41.944 INFO:teuthology.orchestra.run.smithi094.stdout:2517 trunc from 0xe2fee3d to 0x7e8f890 2024-09-21T22:50:41.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:41.945 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:41.941+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:41.968 INFO:teuthology.orchestra.run.smithi094.stdout:2518 read 0x7187e89 thru 0x718edb1 (0x6f29 bytes) 2024-09-21T22:50:41.968 INFO:teuthology.orchestra.run.smithi094.stdout:2519 punch from 0x34ee4f7 to 0x34fb0fb, (0xcc04 bytes) 2024-09-21T22:50:41.968 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:41.968 INFO:teuthology.orchestra.run.smithi094.stdout:2521 read 0x3fe9010 thru 0x3ff607e (0xd06f bytes) 2024-09-21T22:50:41.968 INFO:teuthology.orchestra.run.smithi094.stdout:2522 write 0x51afcb5 thru 0x51bb82d (0xbb79 bytes) 2024-09-21T22:50:42.044 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:42.041+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:42.047 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:50:42.047 INFO:teuthology.orchestra.run.smithi094.stdout:2523 write 0x38e5e87 thru 0x38ec63c (0x67b6 bytes) 2024-09-21T22:50:42.063 INFO:teuthology.orchestra.run.smithi094.stdout:2524 write 0x56fb576 thru 0x56fc0ec (0xb77 bytes) 2024-09-21T22:50:42.078 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:42.078 INFO:teuthology.orchestra.run.smithi094.stdout:2526 read 0x3821474 thru 0x3828ae3 (0x7670 bytes) 2024-09-21T22:50:42.088 INFO:teuthology.orchestra.run.smithi094.stdout:2527 trunc from 0x7e8f890 to 0xd9889a9 2024-09-21T22:50:42.094 INFO:teuthology.orchestra.run.smithi094.stdout:2528 read 0x560b7ad thru 0x560bb4e (0x3a2 bytes) 2024-09-21T22:50:42.094 INFO:teuthology.orchestra.run.smithi094.stdout:2529 write 0xbcf02fc thru 0xbcf0803 (0x508 bytes) 2024-09-21T22:50:42.098 INFO:teuthology.orchestra.run.smithi094.stdout:2532 write 0x3a0c977 thru 0x3a18660 (0xbcea bytes) 2024-09-21T22:50:42.108 INFO:teuthology.orchestra.run.smithi094.stdout:2533 write 0x4594e23 thru 0x4595fb6 (0x1194 bytes) 2024-09-21T22:50:42.115 INFO:teuthology.orchestra.run.smithi094.stdout:2534 read 0xa987a9b thru 0xa988076 (0x5dc bytes) 2024-09-21T22:50:42.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:42.132 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:42.132 INFO:teuthology.orchestra.run.smithi094.stdout:2536 trunc from 0xd9889a9 to 0x2f5f876 2024-09-21T22:50:42.145 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:42.140+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:42.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:42.209 INFO:teuthology.orchestra.run.smithi094.stdout:2537 write 0x6780711 thru 0x678d577 (0xce67 bytes) 2024-09-21T22:50:42.221 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:42.221 INFO:teuthology.orchestra.run.smithi094.stdout:2539 read 0x195e018 thru 0x196234f (0x4338 bytes) 2024-09-21T22:50:42.221 INFO:teuthology.orchestra.run.smithi094.stdout:2540 write 0x4e80441 thru 0x4e832e4 (0x2ea4 bytes) 2024-09-21T22:50:42.233 INFO:teuthology.orchestra.run.smithi094.stdout:2542 punch from 0x354e8d9 to 0x3558591, (0x9cb8 bytes) 2024-09-21T22:50:42.233 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:42.233 INFO:teuthology.orchestra.run.smithi094.stdout:2544 writesame 0x4b99fcf thru 0x4b9a108 data_size 0x1(0x13a bytes) 2024-09-21T22:50:42.238 INFO:teuthology.orchestra.run.smithi094.stdout:2545 read 0x33cbf36 thru 0x33cccdd (0xda8 bytes) 2024-09-21T22:50:42.238 INFO:teuthology.orchestra.run.smithi094.stdout:2546 read 0x4781116 thru 0x4786ebb (0x5da6 bytes) 2024-09-21T22:50:42.238 INFO:teuthology.orchestra.run.smithi094.stdout:2547 read 0x4fe30f2 thru 0x4feac8f (0x7b9e bytes) 2024-09-21T22:50:42.246 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:42.240+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:42.255 INFO:teuthology.orchestra.run.smithi094.stdout:2548 writesame 0xcf89946 thru 0xcf99474 data_size 0x1(0xfb2f bytes) 2024-09-21T22:50:42.255 INFO:teuthology.orchestra.run.smithi094.stdout:2549 write 0x826a91c thru 0x8276031 (0xb716 bytes) 2024-09-21T22:50:42.267 INFO:teuthology.orchestra.run.smithi094.stdout:2550 write 0x56931e3 thru 0x5697442 (0x4260 bytes) 2024-09-21T22:50:42.278 INFO:teuthology.orchestra.run.smithi094.stdout:2551 writesame 0x861095a thru 0x861bdac data_size 0x35(0xb453 bytes) 2024-09-21T22:50:42.291 INFO:teuthology.orchestra.run.smithi094.stdout:2553 writesame 0x8de20a thru 0x8df478 data_size 0xd(0x126f bytes) 2024-09-21T22:50:42.317 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:42.317 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:42.321 INFO:teuthology.orchestra.run.smithi094.stdout:2554 read 0x9883dd5 thru 0x988ed19 (0xaf45 bytes) 2024-09-21T22:50:42.321 INFO:teuthology.orchestra.run.smithi094.stdout:2556 clone 13 order 18 su 2048 sc 10 2024-09-21T22:50:42.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:42.381 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:42.381+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:42.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.51143914, 2024-09-21T22:50:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12314 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3272704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3724831450000003, 2024-09-21T22:50:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048771374999999999, 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12314 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074252+0000", 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074253+0000", 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.58e-07 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074255+0000", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6919999999999998e-06 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074263+0000", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6590000000000007e-06 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074269+0000", 2024-09-21T22:50:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0499999999999997e-06 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.122919+0000", 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048650021000000002 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.122962+0000", 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3146999999999999e-05 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123024+0000", 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1347999999999996e-05 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12315 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3313664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3724584869999994, 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049012636999999998, 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12315 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.074277+0000", 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077706+0000", 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034286859999999998 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077722+0000", 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5849e-05 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077735+0000", 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3149000000000001e-05 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.077743+0000", 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6629999999999999e-06 2024-09-21T22:50:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123072+0000", 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045329031999999998 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123161+0000", 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8658000000000004e-05 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.123290+0000", 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012860000000000001 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12367 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 2826240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3647721629999996, 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049023430999999999, 2024-09-21T22:50:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12367 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081963+0000", 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081964+0000", 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4499999999999998e-07 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081970+0000", 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0229999999999998e-06 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081974+0000", 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7419999999999997e-06 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081977+0000", 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.021e-06 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130764+0000", 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048786325999999998 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130855+0000", 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1533000000000006e-05 2024-09-21T22:50:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.130987+0000", 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013154099999999999 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12368 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 2867200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.364754306, 2024-09-21T22:50:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049220224999999999, 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12368 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.081981+0000", 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085829+0000", 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003847984 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085841+0000", 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1598e-05 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085848+0000", 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7200000000000006e-06 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.085853+0000", 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8069999999999999e-06 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131033+0000", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045179929000000001 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131104+0000", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1101999999999998e-05 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.131201+0000", 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7084999999999996e-05 2024-09-21T22:50:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12383 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3481600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3606889869999996, 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048843009, 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12383 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086046+0000", 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086047+0000", 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086048+0000", 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086051+0000", 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.343e-06 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086054+0000", 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8619999999999999e-06 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134743+0000", 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048689112 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134806+0000", 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2904000000000006e-05 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134889+0000", 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3349000000000004e-05 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12384 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3522560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.360677956, 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049097057999999999, 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12384 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.086058+0000", 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089645+0000", 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035875820000000002 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089649+0000", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089662+0000", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2323e-05 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.089669+0000", 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2559999999999996e-06 2024-09-21T22:50:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.134935+0000", 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045265908000000001 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.135040+0000", 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105096 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.135155+0000", 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114608 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12447 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3485696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3487112660000005, 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048875292000000001, 2024-09-21T22:50:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12447 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098024+0000", 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098025+0000", 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0499999999999999e-07 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098027+0000", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.655e-06 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098043+0000", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5753999999999999e-05 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098048+0000", 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2739999999999998e-06 2024-09-21T22:50:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146679+0000", 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048630975 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146785+0000", 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010550100000000001 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146899+0000", 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114728 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12448 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3486796569999999, 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049045176000000003, 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12448 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098056+0000", 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9999999999999998e-07 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098059+0000", 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.672e-06 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098066+0000", 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-06 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098071+0000", 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1590000000000001e-06 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.146944+0000", 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048873460000000001 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147010+0000", 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5673000000000006e-05 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147101+0000", 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0941999999999998e-05 2024-09-21T22:50:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12449 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3567616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3486577850000003, 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049294103999999998, 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12449 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098078+0000", 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.15e-07 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098080+0000", 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098087+0000", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2670000000000003e-06 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098091+0000", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8640000000000004e-06 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147153+0000", 2024-09-21T22:50:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049061950999999999 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147259+0000", 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106026 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147372+0000", 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011244899999999999 2024-09-21T22:50:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12450 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3608576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3486373799999996, 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049475004000000003, 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12450 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098098+0000", 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098099+0000", 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6400000000000003e-07 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098101+0000", 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0980000000000001e-06 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098107+0000", 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4799999999999998e-06 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098112+0000", 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0170000000000004e-06 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147416+0000", 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049304065000000001 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147482+0000", 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6202000000000001e-05 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147573+0000", 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0678000000000002e-05 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12451 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3486020750000005, 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049716228000000001, 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12451 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098133+0000", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098134+0000", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0600000000000001e-07 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098136+0000", 2024-09-21T22:50:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098140+0000", 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098144+0000", 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2310000000000001e-06 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147626+0000", 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049482019000000002 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147732+0000", 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010628600000000001 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147850+0000", 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117652 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12452 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3690496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485860489999997, 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049897555000000003, 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12452 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098149+0000", 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098150+0000", 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-07 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098157+0000", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3749999999999997e-06 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098162+0000", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7520000000000004e-06 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098165+0000", 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0599999999999999e-06 2024-09-21T22:50:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147894+0000", 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049729168999999997 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.147958+0000", 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3841000000000001e-05 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148047+0000", 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9090000000000003e-05 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12453 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3731456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485664269999997, 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050146668999999998, 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12453 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098169+0000", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.03e-07 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098171+0000", 2024-09-21T22:50:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3969999999999999e-06 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098176+0000", 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0139999999999998e-06 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098179+0000", 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148098+0000", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049919193000000001 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148198+0000", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100271 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148316+0000", 2024-09-21T22:50:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117296 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12454 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3772416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485530029999993, 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050328201000000003, 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12454 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098182+0000", 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098183+0000", 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5699999999999999e-07 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098184+0000", 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739999999999999e-06 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098187+0000", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.427e-06 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098190+0000", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0649999999999999e-06 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148360+0000", 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050169381999999998 2024-09-21T22:50:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148425+0000", 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4916e-05 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148511+0000", 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.598e-05 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12455 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3813376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485416039999993, 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050678511000000002, 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12455 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098194+0000", 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5699999999999999e-07 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098195+0000", 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.215e-06 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098199+0000", 2024-09-21T22:50:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.428e-06 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098202+0000", 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148563+0000", 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050361446999999997 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148670+0000", 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107192 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148872+0000", 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020216400000000001 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12456 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3854336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485303969999993, 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050867531000000001, 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12456 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098205+0000", 2024-09-21T22:50:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5899999999999998e-07 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098207+0000", 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.381e-06 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098210+0000", 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3400000000000002e-06 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098213+0000", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7949999999999999e-06 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148918+0000", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050705061000000003 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.148984+0000", 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5808000000000004e-05 2024-09-21T22:50:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149073+0000", 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8887000000000005e-05 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12457 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 3895296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3485190669999998, 2024-09-21T22:50:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051131854999999997, 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12457 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.098216+0000", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101657+0000", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00344014 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101661+0000", 2024-09-21T22:50:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6260000000000003e-06 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101675+0000", 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3793e-05 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101683+0000", 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1629999999999996e-06 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149123+0000", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047440319000000002 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149230+0000", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010642 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.149348+0000", 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011839400000000001 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12463 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 4141056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3448819109999999, 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048991664999999997, 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12463 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101854+0000", 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3000000000000001e-07 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101856+0000", 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.182e-06 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101863+0000", 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5429999999999999e-06 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101868+0000", 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.023e-06 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150621+0000", 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048752789999999997 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150711+0000", 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150845+0000", 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013396600000000001 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:12464 3.f 3:f059107f:::rbd_data.111ea12b32db.0000000000000002:c [read 4182016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3448604559999993, 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212819999999997, 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 12464 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101875+0000", 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8500000000000002e-07 2024-09-21T22:50:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101882+0000", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0550000000000002e-06 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101890+0000", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.101897+0000", 2024-09-21T22:50:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7710000000000001e-06 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150884+0000", 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048987902 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.150971+0000", 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6315000000000003e-05 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:33.151088+0000", 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011707200000000001 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:42.550 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:42.551 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:42.480+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:42.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:42.552 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:42.544+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:42.592 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:42.593 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:42.609 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:42.609 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:42.625 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:42.626 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:42.642 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:42.642 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:42.652 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:42.648+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:42.660 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:42.660 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:42.678 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:42.678 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:42.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:42.697 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:42.716 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:42.716 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:42.728 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:42.728 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:42.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:50:42.777 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:42.772+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:42.854 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:42.848+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:42.860 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:42.861 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:42.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:50:42.955 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:42.948+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:42.991 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:42.991 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:43.056 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:43.048+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:43.142 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:43.142 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.156 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:43.152+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:43.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:43.186 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone12 from 0xcf99475 (overlap 0x2f5f876) to 0x979817 2024-09-21T22:50:43.258 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:43.257+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:43.339 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:43.339 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.358 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:43.352+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:43.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:43.459 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:43.452+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:43.560 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:43.556+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:43.589 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:43.589 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:43.696 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:43.692+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:43.731 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T22:50:43.737 INFO:teuthology.orchestra.run.smithi094.stdout:2559 read 0x6ce5842 thru 0x6cf1ee4 (0xc6a3 bytes) 2024-09-21T22:50:43.738 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.738 INFO:teuthology.orchestra.run.smithi094.stdout:2563 writesame 0x709d52d thru 0x709d762 data_size 0x1(0x236 bytes) 2024-09-21T22:50:43.749 INFO:teuthology.orchestra.run.smithi094.stdout:2564 write 0xcb6a8be thru 0xcb6c6e4 (0x1e27 bytes) 2024-09-21T22:50:43.762 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:43.756+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:43.771 INFO:teuthology.orchestra.run.smithi094.stdout:2565 write 0xd09ce77 thru 0xd09e4d2 (0x165c bytes) 2024-09-21T22:50:43.774 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:43.774 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.783 INFO:teuthology.orchestra.run.smithi094.stdout:2567 read 0x1a91006 thru 0x1a99601 (0x85fc bytes) 2024-09-21T22:50:43.783 INFO:teuthology.orchestra.run.smithi094.stdout:2568 write 0xb89f5c6 thru 0xb8ac442 (0xce7d bytes) 2024-09-21T22:50:43.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:43.804 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.805 INFO:teuthology.orchestra.run.smithi094.stdout:2571 write 0x2a13c92 thru 0x2a14b04 (0xe73 bytes) 2024-09-21T22:50:43.815 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.815 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.815 INFO:teuthology.orchestra.run.smithi094.stdout:2576 read 0x89c44fc thru 0x89c718f (0x2c94 bytes) 2024-09-21T22:50:43.815 INFO:teuthology.orchestra.run.smithi094.stdout:2578 read 0x80d5395 thru 0x80e417f (0xedeb bytes) 2024-09-21T22:50:43.815 INFO:teuthology.orchestra.run.smithi094.stdout:2579 write 0x1160137 thru 0x11649b2 (0x487c bytes) 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:2582 read 0x9c8bfa7 thru 0x9c9165a (0x56b4 bytes) 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:2585 read 0x5f908b4 thru 0x5f997d2 (0x8f1f bytes) 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:2586 punch from 0xb0d219a to 0xb0e16d3, (0xf539 bytes) 2024-09-21T22:50:43.834 INFO:teuthology.orchestra.run.smithi094.stdout:2588 writesame 0xa12438 thru 0xa166e5 data_size 0xf(0x42ae bytes) 2024-09-21T22:50:43.863 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:43.856+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:43.886 INFO:teuthology.orchestra.run.smithi094.stdout:2590 read 0x2805bbc thru 0x2814686 (0xeacb bytes) 2024-09-21T22:50:43.887 INFO:teuthology.orchestra.run.smithi094.stdout:2591 read 0x33ac33a thru 0x33ae8b0 (0x2577 bytes) 2024-09-21T22:50:43.887 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:43.887 INFO:teuthology.orchestra.run.smithi094.stdout:2593 read 0x38a41d1 thru 0x38ad75f (0x958f bytes) 2024-09-21T22:50:43.887 INFO:teuthology.orchestra.run.smithi094.stdout:2594 write 0x2dbca0d thru 0x2dc333f (0x6933 bytes) 2024-09-21T22:50:43.964 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:43.956+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:43.966 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:43.966 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:43.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:43.983 INFO:teuthology.orchestra.run.smithi094.stdout:2596 write 0x38decf9 thru 0x38e6943 (0x7c4b bytes) 2024-09-21T22:50:44.003 INFO:teuthology.orchestra.run.smithi094.stdout:2597 read 0x18496bb thru 0x18499b3 (0x2f9 bytes) 2024-09-21T22:50:44.003 INFO:teuthology.orchestra.run.smithi094.stdout:2598 write 0x93c867 thru 0x94c61f (0xfdb9 bytes) 2024-09-21T22:50:44.064 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:44.060+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:44.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:50:44.099 INFO:teuthology.orchestra.run.smithi094.stdout:2600 trunc from 0xd09e4d3 to 0xd705e7f 2024-09-21T22:50:44.106 INFO:teuthology.orchestra.run.smithi094.stdout:2602 write 0x3ffb207 thru 0x4006d4f (0xbb49 bytes) 2024-09-21T22:50:44.123 INFO:teuthology.orchestra.run.smithi094.stdout:2605 write 0xadbf0d6 thru 0xadca3df (0xb30a bytes) 2024-09-21T22:50:44.148 INFO:teuthology.orchestra.run.smithi094.stdout:2607 write 0xe764b3a thru 0xe76ab85 (0x604c bytes) 2024-09-21T22:50:44.154 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:44.155 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:44.164 INFO:teuthology.orchestra.run.smithi094.stdout:2608 read 0x1075cf5 thru 0x107c7a3 (0x6aaf bytes) 2024-09-21T22:50:44.165 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:44.161+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:44.165 INFO:teuthology.orchestra.run.smithi094.stdout:2609 write 0x769856a thru 0x76a0ddd (0x8874 bytes) 2024-09-21T22:50:44.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:44.233 INFO:teuthology.orchestra.run.smithi094.stdout:2610 punch from 0x81b56aa to 0x81c329c, (0xdbf2 bytes) 2024-09-21T22:50:44.233 INFO:teuthology.orchestra.run.smithi094.stdout:2612 write 0xd0d034f thru 0xd0d4b49 (0x47fb bytes) 2024-09-21T22:50:44.249 INFO:teuthology.orchestra.run.smithi094.stdout:2615 writesame 0xa331581 thru 0xa338be3 data_size 0x1(0x7663 bytes) 2024-09-21T22:50:44.265 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:44.265+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:44.272 INFO:teuthology.orchestra.run.smithi094.stdout:2616 read 0x28bce30 thru 0x28c65df (0x97b0 bytes) 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:2617 read 0x8e9b1ab thru 0x8ea0fcf (0x5e25 bytes) 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:2618 punch from 0x38ac7d to 0x38f5b0, (0x4933 bytes) 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:2619 read 0x4a6333 thru 0x4ad807 (0x74d5 bytes) 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:2620 read 0x1bebd0f thru 0x1bf0ed6 (0x51c8 bytes) 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.273 INFO:teuthology.orchestra.run.smithi094.stdout:2623 write 0x19e49dd thru 0x19ebddf (0x7403 bytes) 2024-09-21T22:50:44.296 INFO:teuthology.orchestra.run.smithi094.stdout:2624 punch from 0xda2f66c to 0xda3583d, (0x61d1 bytes) 2024-09-21T22:50:44.296 INFO:teuthology.orchestra.run.smithi094.stdout:2625 read 0xdb809d thru 0xdb95e9 (0x154d bytes) 2024-09-21T22:50:44.296 INFO:teuthology.orchestra.run.smithi094.stdout:2626 punch from 0xaded43 to 0xae860e, (0x98cb bytes) 2024-09-21T22:50:44.296 INFO:teuthology.orchestra.run.smithi094.stdout:2627 write 0x228a152 thru 0x228d7dc (0x368b bytes) 2024-09-21T22:50:44.351 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:44.351 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:44.364 INFO:teuthology.orchestra.run.smithi094.stdout:2628 write 0x63f0a09 thru 0x63fb0e7 (0xa6df bytes) 2024-09-21T22:50:44.367 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:44.367 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:44.365+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:44.376 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.376 INFO:teuthology.orchestra.run.smithi094.stdout:2631 write 0x447223 thru 0x44c74f (0x552d bytes) 2024-09-21T22:50:44.412 INFO:teuthology.orchestra.run.smithi094.stdout:2632 read 0xb0681d0 thru 0xb076327 (0xe158 bytes) 2024-09-21T22:50:44.412 INFO:teuthology.orchestra.run.smithi094.stdout:2633 read 0x2dbe7e2 thru 0x2dc087d (0x209c bytes) 2024-09-21T22:50:44.412 INFO:teuthology.orchestra.run.smithi094.stdout:2634 writesame 0x71760a1 thru 0x717c69b data_size 0x1(0x65fb bytes) 2024-09-21T22:50:44.427 INFO:teuthology.orchestra.run.smithi094.stdout:2636 writesame 0x86aa066 thru 0x86b5cc4 data_size 0x7(0xbc5f bytes) 2024-09-21T22:50:44.467 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:44.465+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:44.486 INFO:teuthology.orchestra.run.smithi094.stdout:2637 write 0x404dcf2 thru 0x404f55c (0x186b bytes) 2024-09-21T22:50:44.488 INFO:teuthology.orchestra.run.smithi094.stdout:2638 write 0x485536d thru 0x48568a1 (0x1535 bytes) 2024-09-21T22:50:44.495 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.495 INFO:teuthology.orchestra.run.smithi094.stdout:2643 punch from 0x407df34 to 0x4081180, (0x324c bytes) 2024-09-21T22:50:44.495 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.495 INFO:teuthology.orchestra.run.smithi094.stdout:2645 punch from 0xa6aee44 to 0xa6b6f39, (0x80f5 bytes) 2024-09-21T22:50:44.500 INFO:teuthology.orchestra.run.smithi094.stdout:2646 write 0xed02f0c thru 0xed0c91d (0x9a12 bytes) 2024-09-21T22:50:44.517 INFO:teuthology.orchestra.run.smithi094.stdout:2647 write 0x35f25cf thru 0x35f8ac2 (0x64f4 bytes) 2024-09-21T22:50:44.521 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:44.522 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:44.530 INFO:teuthology.orchestra.run.smithi094.stdout:2648 read 0xbee65f thru 0xbeee86 (0x828 bytes) 2024-09-21T22:50:44.531 INFO:teuthology.orchestra.run.smithi094.stdout:2649 read 0xaace9e9 thru 0xaade81a (0xfe32 bytes) 2024-09-21T22:50:44.531 INFO:teuthology.orchestra.run.smithi094.stdout:2651 read 0xb14b0b3 thru 0xb14d24a (0x2198 bytes) 2024-09-21T22:50:44.531 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.531 INFO:teuthology.orchestra.run.smithi094.stdout:2653 read 0x70a5730 thru 0x70ab70a (0x5fdb bytes) 2024-09-21T22:50:44.532 INFO:teuthology.orchestra.run.smithi094.stdout:2654 write 0xc4d27c5 thru 0xc4e2071 (0xf8ad bytes) 2024-09-21T22:50:44.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:44.552 INFO:teuthology.orchestra.run.smithi094.stdout:2655 trunc from 0xed0c91e to 0xde8cb4d 2024-09-21T22:50:44.567 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:44.565+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:44.572 INFO:teuthology.orchestra.run.smithi094.stdout:2656 write 0x225b180 thru 0x225d72f (0x25b0 bytes) 2024-09-21T22:50:44.600 INFO:teuthology.orchestra.run.smithi094.stdout:2657 punch from 0x9a03db5 to 0x9a0efc5, (0xb210 bytes) 2024-09-21T22:50:44.614 INFO:teuthology.orchestra.run.smithi094.stdout:2658 trunc from 0xde8cb4d to 0x21ad8de 2024-09-21T22:50:44.668 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:44.660+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:44.682 INFO:teuthology.orchestra.run.smithi094.stdout:2661 punch from 0x18c7bf7 to 0x18c7f94, (0x39d bytes) 2024-09-21T22:50:44.682 INFO:teuthology.orchestra.run.smithi094.stdout:2664 read 0xdf6ebe thru 0xe03fe3 (0xd126 bytes) 2024-09-21T22:50:44.682 INFO:teuthology.orchestra.run.smithi094.stdout:2665 read 0x3f9496 thru 0x4004fa (0x7065 bytes) 2024-09-21T22:50:44.684 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:44.684 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:44.702 INFO:teuthology.orchestra.run.smithi094.stdout:2666 write 0xd171506 thru 0xd1807a0 (0xf29b bytes) 2024-09-21T22:50:44.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:44.719 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.720 INFO:teuthology.orchestra.run.smithi094.stdout:2669 read 0x21d14ab thru 0x21d9d8a (0x88e0 bytes) 2024-09-21T22:50:44.721 INFO:teuthology.orchestra.run.smithi094.stdout:2670 write 0x70c4816 thru 0x70c7ab3 (0x329e bytes) 2024-09-21T22:50:44.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:44.734 INFO:teuthology.orchestra.run.smithi094.stdout:2671 read 0x885bd76 thru 0x886492f (0x8bba bytes) 2024-09-21T22:50:44.735 INFO:teuthology.orchestra.run.smithi094.stdout:2672 write 0x5d41365 thru 0x5d4a403 (0x909f bytes) 2024-09-21T22:50:44.749 INFO:teuthology.orchestra.run.smithi094.stdout:2673 punch from 0x209d046 to 0x20a77b2, (0xa76c bytes) 2024-09-21T22:50:44.758 INFO:teuthology.orchestra.run.smithi094.stdout:2676 trunc from 0xd1807a1 to 0x5ae1787 2024-09-21T22:50:44.769 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:44.764+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:44.780 INFO:teuthology.orchestra.run.smithi094.stdout:2677 read 0x35e8b28 thru 0x35efe6f (0x7348 bytes) 2024-09-21T22:50:44.780 INFO:teuthology.orchestra.run.smithi094.stdout:2678 write 0x297bbe2 thru 0x29884fb (0xc91a bytes) 2024-09-21T22:50:44.800 INFO:teuthology.orchestra.run.smithi094.stdout:2679 trunc from 0x5ae1787 to 0x97fae0c 2024-09-21T22:50:44.806 INFO:teuthology.orchestra.run.smithi094.stdout:2680 read 0x78be65e thru 0x78c7d0a (0x96ad bytes) 2024-09-21T22:50:44.806 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.806 INFO:teuthology.orchestra.run.smithi094.stdout:2683 write 0x3ff831b thru 0x3ffa509 (0x21ef bytes) 2024-09-21T22:50:44.816 INFO:teuthology.orchestra.run.smithi094.stdout:2685 trunc from 0x97fae0c to 0x94f9d8f 2024-09-21T22:50:44.833 INFO:teuthology.orchestra.run.smithi094.stdout:2686 writesame 0xc770e93 thru 0xc777fa0 data_size 0x1d(0x710e bytes) 2024-09-21T22:50:44.841 INFO:teuthology.orchestra.run.smithi094.stdout:2687 write 0xdf26613 thru 0xdf31f8c (0xb97a bytes) 2024-09-21T22:50:44.852 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:44.852 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:44.859 INFO:teuthology.orchestra.run.smithi094.stdout:2688 punch from 0x2c8e221 to 0x2c92dc5, (0x4ba4 bytes) 2024-09-21T22:50:44.859 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.862 INFO:teuthology.orchestra.run.smithi094.stdout:2691 trunc from 0xdf31f8d to 0xb7963bc 2024-09-21T22:50:44.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:44.871 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:44.869+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:44.885 INFO:teuthology.orchestra.run.smithi094.stdout:2692 read 0x7b133a2 thru 0x7b202df (0xcf3e bytes) 2024-09-21T22:50:44.885 INFO:teuthology.orchestra.run.smithi094.stdout:2693 write 0x945da59 thru 0x945ebf0 (0x1198 bytes) 2024-09-21T22:50:44.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:44.887 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:44.891 INFO:teuthology.orchestra.run.smithi094.stdout:2696 read 0x578b2d7 thru 0x5797a8a (0xc7b4 bytes) 2024-09-21T22:50:44.891 INFO:teuthology.orchestra.run.smithi094.stdout:2697 punch from 0x54616ba to 0x5465ea1, (0x47e7 bytes) 2024-09-21T22:50:44.903 INFO:teuthology.orchestra.run.smithi094.stdout:2699 writesame 0xebcd7b3 thru 0xebd5105 data_size 0x77(0x7953 bytes) 2024-09-21T22:50:44.904 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:44.904 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:44.904 INFO:teuthology.orchestra.run.smithi094.stdout:2700 read 0x39c6497 thru 0x39c8c9b (0x2805 bytes) 2024-09-21T22:50:44.904 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.905 INFO:teuthology.orchestra.run.smithi094.stdout:2704 read 0x71f9160 thru 0x71ff520 (0x63c1 bytes) 2024-09-21T22:50:44.905 INFO:teuthology.orchestra.run.smithi094.stdout:2705 write 0x62a1345 thru 0x62aa0b0 (0x8d6c bytes) 2024-09-21T22:50:44.922 INFO:teuthology.orchestra.run.smithi094.stdout:2706 writesame 0x60d15ac thru 0x60d791c data_size 0x1(0x6371 bytes) 2024-09-21T22:50:44.925 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:44.925 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:44.943 INFO:teuthology.orchestra.run.smithi094.stdout:2707 writesame 0x65deb1e thru 0x65ed454 data_size 0x7(0xe937 bytes) 2024-09-21T22:50:44.946 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:44.959 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:44.972 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:44.969+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:44.974 INFO:teuthology.orchestra.run.smithi094.stdout:2710 trunc from 0xebd5106 to 0x3db993a 2024-09-21T22:50:44.974 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:44.998 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:44.998 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:45.016 INFO:teuthology.orchestra.run.smithi094.stdout:2711 write 0x939b76b thru 0x939ffc9 (0x485f bytes) 2024-09-21T22:50:45.018 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:45.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:45.036 INFO:teuthology.orchestra.run.smithi094.stdout:2712 write 0xecb7ce thru 0xed84a3 (0xccd6 bytes) 2024-09-21T22:50:45.044 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:45.044 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:45.055 INFO:teuthology.orchestra.run.smithi094.stdout:2713 writesame 0x5a92044 thru 0x5a9bc38 data_size 0x5(0x9bf5 bytes) 2024-09-21T22:50:45.068 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:50:45.072 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:45.069+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:45.080 INFO:teuthology.orchestra.run.smithi094.stdout:2714 writesame 0x5b2b37 thru 0x5c25a9 data_size 0x1(0xfa73 bytes) 2024-09-21T22:50:45.181 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:45.177+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6462 3.2 3:44ccee00:::rbd_data.111ebc806776.000000000000001a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.420137079, 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.08074365, 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6462 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762174+0000", 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3029999999999998e-06 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762186+0000", 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.136e-05 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762217+0000", 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0663000000000001e-05 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762232+0000", 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5099999999999999e-05 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762388+0000", 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000156362 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762632+0000", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243995 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.777316+0000", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014684262999999999 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842690+0000", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065373938000000006 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842722+0000", 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2283999999999998e-05 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842825+0000", 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102192 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842916+0000", 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1189999999999999e-05 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13534 3.1e 3:7f270d40:::rbd_data.111e4c6c7d62.000000000000022a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1556521039999996, 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094102410000001, 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13534 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026659+0000", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.544e-06 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026666+0000", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0679999999999999e-06 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026681+0000", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4758e-05 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026720+0000", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9595999999999998e-05 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026822+0000", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101512 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026833+0000", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1698999999999999e-05 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.031773+0000", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049400070000001 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032053+0000", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027928499999999999 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032241+0000", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000188448 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.034179+0000", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001938214 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035722+0000", 2024-09-21T22:50:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001542762 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035951+0000", 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000229232 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.036067+0000", 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116116 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19015 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 737280~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714546+0000", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4677634620000002, 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046090820999999997, 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19015 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714546+0000", 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714546+0000", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714547+0000", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714552+0000", 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7199999999999997e-06 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714568+0000", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6704000000000001e-05 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714583+0000", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5156e-05 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760447+0000", 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.045863956999999997 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760549+0000", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102005 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760637+0000", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7295999999999999e-05 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19016 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 751616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714600+0000", 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.467709132, 2024-09-21T22:50:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046243666000000003, 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19016 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714600+0000", 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714600+0000", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714601+0000", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714606+0000", 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0200000000000002e-06 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714623+0000", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6535999999999999e-05 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714636+0000", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3327e-05 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760686+0000", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046049700999999998 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760769+0000", 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3240999999999998e-05 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760844+0000", 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4810999999999999e-05 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19017 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 761856~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714673+0000", 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4676362630000002, 2024-09-21T22:50:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046400221999999998, 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19017 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714673+0000", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714673+0000", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714674+0000", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.443e-06 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714680+0000", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5140000000000001e-06 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714698+0000", 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8070000000000001e-05 2024-09-21T22:50:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714711+0000", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3373000000000001e-05 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760890+0000", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046178668999999999 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.760984+0000", 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4093999999999997e-05 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761073+0000", 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9059000000000002e-05 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19018 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 772096~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714728+0000", 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4675810600000001, 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046596547000000002, 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19018 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714728+0000", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714728+0000", 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714729+0000", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2270000000000001e-06 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714735+0000", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3770000000000004e-06 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714752+0000", 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7139999999999999e-05 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714765+0000", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3066e-05 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761134+0000", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046369059999999997 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761247+0000", 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112638 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761325+0000", 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8039000000000003e-05 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19019 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 796672~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714781+0000", 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4675286299999999, 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046771103000000001, 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19019 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714781+0000", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714781+0000", 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714782+0000", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.33e-06 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714787+0000", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.933e-06 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714803+0000", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6595999999999998e-05 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714817+0000", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3166999999999999e-05 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761371+0000", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046554521000000001 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761465+0000", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3567999999999996e-05 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761552+0000", 2024-09-21T22:50:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6988000000000001e-05 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19020 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 806912~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714833+0000", 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4674762870000002, 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046932883000000002, 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19020 2024-09-21T22:50:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714833+0000", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714833+0000", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714834+0000", 2024-09-21T22:50:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2160000000000001e-06 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714839+0000", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9899999999999997e-06 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714856+0000", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6727000000000001e-05 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714868+0000", 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2480000000000001e-05 2024-09-21T22:50:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761612+0000", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046743409 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761695+0000", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3487999999999998e-05 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761766+0000", 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0573000000000003e-05 2024-09-21T22:50:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19021 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 831488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714885+0000", 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4674245369999999, 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047126750000000002, 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19021 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714885+0000", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714885+0000", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714886+0000", 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714892+0000", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8830000000000003e-06 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714910+0000", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7846000000000002e-05 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714924+0000", 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4827000000000001e-05 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761810+0000", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046886048999999999 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.761924+0000", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011366000000000001 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762011+0000", 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7365000000000001e-05 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19022 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 841728~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.714942+0000", 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.467366905, 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047276091999999999, 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19022 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714942+0000", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714942+0000", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714943+0000", 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0640000000000001e-06 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714958+0000", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4413000000000001e-05 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714979+0000", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1233000000000002e-05 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.714993+0000", 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3823e-05 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762054+0000", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047060794000000003 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762134+0000", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0315000000000006e-05 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762218+0000", 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4450000000000006e-05 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19023 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 851968~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.715012+0000", 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4672976850000001, 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047445775000000003, 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19023 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715012+0000", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715012+0000", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715013+0000", 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.083e-06 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715018+0000", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7139999999999998e-06 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715036+0000", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715049+0000", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3020000000000001e-05 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762270+0000", 2024-09-21T22:50:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047220913000000003 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762369+0000", 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9298999999999999e-05 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762457+0000", 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8005999999999997e-05 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19024 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 866304~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.715066+0000", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4672437550000001, 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047635312999999999, 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19024 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715066+0000", 2024-09-21T22:50:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715066+0000", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715066+0000", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715072+0000", 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0259999999999998e-06 2024-09-21T22:50:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715088+0000", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6594000000000001e-05 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715101+0000", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3046e-05 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762499+0000", 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047397742999999999 2024-09-21T22:50:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762580+0000", 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1600999999999993e-05 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762701+0000", 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012032 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19025 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 876544~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.715137+0000", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.467172299, 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047834967999999999, 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19025 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715137+0000", 2024-09-21T22:50:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715137+0000", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715138+0000", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.356e-06 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715144+0000", 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:50:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715162+0000", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7589999999999999e-05 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.715175+0000", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.33e-05 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762757+0000", 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047581489999999997 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762878+0000", 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120877 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.762972+0000", 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4325000000000002e-05 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19135 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 614400~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.725290+0000", 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.457019179, 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046302713000000002, 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19135 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.725290+0000", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.725290+0000", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.728530+0000", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032401000000000001 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.728540+0000", 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5750000000000007e-06 2024-09-21T22:50:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.728557+0000", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7513e-05 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.728571+0000", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3602e-05 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.771318+0000", 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042747159 2024-09-21T22:50:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.771455+0000", 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136768 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.771593+0000", 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013799600000000001 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19148 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 808960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729146+0000", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4531629009999998, 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04618676, 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19148 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729146+0000", 2024-09-21T22:50:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729146+0000", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729147+0000", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9299999999999998e-07 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729149+0000", 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5019999999999999e-06 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729157+0000", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-06 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729163+0000", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7200000000000003e-06 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775134+0000", 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.045970313999999998 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775246+0000", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011295599999999999 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775333+0000", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6655000000000003e-05 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19149 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 819200~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729194+0000", 2024-09-21T22:50:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4531147980000001, 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046461339999999997, 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19149 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729194+0000", 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729194+0000", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729195+0000", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729199+0000", 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.495e-06 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729207+0000", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6719999999999993e-06 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729216+0000", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0499999999999997e-06 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775396+0000", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046180134999999997 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775537+0000", 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014017399999999999 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775656+0000", 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011913699999999999 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19150 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 833536~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729226+0000", 2024-09-21T22:50:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4530829880000002, 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046666488999999998, 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19150 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729226+0000", 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729226+0000", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729227+0000", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1399999999999997e-07 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729229+0000", 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.616e-06 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729237+0000", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1520000000000003e-06 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729242+0000", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8499999999999999e-06 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775721+0000", 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046478102 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775801+0000", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0589999999999999e-05 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775893+0000", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1664999999999997e-05 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19151 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 843776~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729250+0000", 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4530595059999998, 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046976250999999997, 2024-09-21T22:50:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19151 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729250+0000", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729250+0000", 2024-09-21T22:50:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729251+0000", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729253+0000", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5579999999999999e-06 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729260+0000", 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5119999999999997e-06 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729265+0000", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7599999999999999e-06 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.775971+0000", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04670564 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776103+0000", 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013237500000000001 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776226+0000", 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000122552 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19152 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 854016~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729287+0000", 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4530219959999999, 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047331192000000001, 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19152 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729287+0000", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729287+0000", 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729288+0000", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3399999999999999e-07 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729291+0000", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729298+0000", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3420000000000002e-06 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729304+0000", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9599999999999997e-06 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776291+0000", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046986653000000003 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776439+0000", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014872900000000001 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776618+0000", 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017909100000000001 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:19153 3.2 3:43724fa6:::rbd_data.111ebb10c685.00000000000000a2:e [read 878592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:41.729311+0000", 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4529978039999998, 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047636297000000001, 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 19153 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729311+0000", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729311+0000", 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729312+0000", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729315+0000", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5749999999999999e-06 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729321+0000", 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6619999999999999e-06 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.729327+0000", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7749999999999998e-06 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776691+0000", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.047364435000000003 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776827+0000", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135306 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:41.776948+0000", 2024-09-21T22:50:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121039 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:45.266 INFO:teuthology.orchestra.run.smithi094.stdout:2715 write 0x7c46d1a thru 0x7c4ae67 (0x414e bytes) 2024-09-21T22:50:45.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:45.268 INFO:teuthology.orchestra.run.smithi094.stdout:2718 read 0x7308b43 thru 0x73115ad (0x8a6b bytes) 2024-09-21T22:50:45.268 INFO:teuthology.orchestra.run.smithi094.stdout:2719 read 0x4de6d25 thru 0x4deabe7 (0x3ec3 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2721 trunc from 0x939ffca to 0xed1a0b5 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2722 read 0x31e941b thru 0x31f0c51 (0x7837 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2725 punch from 0x949f497 to 0x94ac857, (0xd3c0 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2728 punch from 0x6049cc0 to 0x604b6d4, (0x1a14 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2729 read 0x1afeb90 thru 0x1b0856d (0x99de bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2730 trunc from 0xed1a0b5 to 0xac2aad0 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2732 read 0x210db4f thru 0x211b7c0 (0xdc72 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2733 write 0x45d9865 thru 0x45e0ce2 (0x747e bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2734 read 0x7a1182c thru 0x7a14bc0 (0x3395 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2735 punch from 0x3642cb to 0x3739e3, (0xf718 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2736 trunc from 0xac2aad0 to 0xace8cd8 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2737 write 0x4293ee0 thru 0x4293ee4 (0x5 bytes) 2024-09-21T22:50:45.269 INFO:teuthology.orchestra.run.smithi094.stdout:2739 trunc from 0xace8cd8 to 0xe9c27da 2024-09-21T22:50:45.272 INFO:teuthology.orchestra.run.smithi094.stdout:2740 writesame 0x1f4489f thru 0x1f4b582 data_size 0x2e(0x6ce4 bytes) 2024-09-21T22:50:45.274 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:45.268+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:45.348 INFO:teuthology.orchestra.run.smithi094.stdout:2741 write 0x7a3daea thru 0x7a3eecd (0x13e4 bytes) 2024-09-21T22:50:45.353 INFO:teuthology.orchestra.run.smithi094.stdout:2742 read 0x50a6f10 thru 0x50b00f2 (0x91e3 bytes) 2024-09-21T22:50:45.358 INFO:teuthology.orchestra.run.smithi094.stdout:2743 trunc from 0xe9c27da to 0xa9a774a 2024-09-21T22:50:45.365 INFO:teuthology.orchestra.run.smithi094.stdout:2744 punch from 0xcde1bd to 0xcec791, (0xe5d4 bytes) 2024-09-21T22:50:45.365 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.365 INFO:teuthology.orchestra.run.smithi094.stdout:2746 read 0x24efb10 thru 0x24f3167 (0x3658 bytes) 2024-09-21T22:50:45.374 INFO:teuthology.orchestra.run.smithi094.stdout:2747 write 0xd2effa1 thru 0xd2fac60 (0xacc0 bytes) 2024-09-21T22:50:45.375 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:45.373+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:45.390 INFO:teuthology.orchestra.run.smithi094.stdout:2748 read 0xa837dc3 thru 0xa83a9a9 (0x2be7 bytes) 2024-09-21T22:50:45.401 INFO:teuthology.orchestra.run.smithi094.stdout:2750 trunc from 0xd2fac61 to 0x5110ff1 2024-09-21T22:50:45.426 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:45.426 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:45.426 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:45.426 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:45.438 INFO:teuthology.orchestra.run.smithi094.stdout:2751 trunc from 0x5110ff1 to 0x133953f 2024-09-21T22:50:45.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:45.475 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:45.468+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:45.504 INFO:teuthology.orchestra.run.smithi094.stdout:2752 write 0x7632dd6 thru 0x763fd0a (0xcf35 bytes) 2024-09-21T22:50:45.523 INFO:teuthology.orchestra.run.smithi094.stdout:2755 write 0x6459b9c thru 0x646276f (0x8bd4 bytes) 2024-09-21T22:50:45.547 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.547 INFO:teuthology.orchestra.run.smithi094.stdout:2757 write 0x2b6669f thru 0x2b75da4 (0xf706 bytes) 2024-09-21T22:50:45.571 INFO:teuthology.orchestra.run.smithi094.stdout:2758 read 0x415d567 thru 0x4168bd0 (0xb66a bytes) 2024-09-21T22:50:45.571 INFO:teuthology.orchestra.run.smithi094.stdout:2759 writesame 0x3606514 thru 0x360c805 data_size 0x22(0x62f2 bytes) 2024-09-21T22:50:45.576 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:45.573+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:45.603 INFO:teuthology.orchestra.run.smithi094.stdout:2760 write 0xa445eed thru 0xa44f5a3 (0x96b7 bytes) 2024-09-21T22:50:45.609 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:45.609 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:45.609 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:45.609 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:45.609 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:45.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:2761 trunc from 0xa44f5a4 to 0x26b11bb 2024-09-21T22:50:45.667 INFO:teuthology.orchestra.run.smithi094.stdout:2764 read 0x147e725 thru 0x1485258 (0x6b34 bytes) 2024-09-21T22:50:45.704 INFO:teuthology.orchestra.run.smithi094.stdout:2766 write 0xeb3cb39 thru 0xeb48e86 (0xc34e bytes) 2024-09-21T22:50:45.713 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:45.713+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:45.730 INFO:teuthology.orchestra.run.smithi094.stdout:2767 writesame 0xbbd3ec7 thru 0xbbd6613 data_size 0x1(0x274d bytes) 2024-09-21T22:50:45.746 INFO:teuthology.orchestra.run.smithi094.stdout:2768 punch from 0x229241f to 0x2293fce, (0x1baf bytes) 2024-09-21T22:50:45.746 INFO:teuthology.orchestra.run.smithi094.stdout:2769 write 0xbce8852 thru 0xbce9254 (0xa03 bytes) 2024-09-21T22:50:45.748 INFO:teuthology.orchestra.run.smithi094.stdout:2770 write 0xac7c1b9 thru 0xac7fcf2 (0x3b3a bytes) 2024-09-21T22:50:45.766 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.766 INFO:teuthology.orchestra.run.smithi094.stdout:2772 writesame 0xcda9eb6 thru 0xcdb9bf0 data_size 0x93(0xfd3b bytes) 2024-09-21T22:50:45.778 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:45.772+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:45.786 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.786 INFO:teuthology.orchestra.run.smithi094.stdout:2774 writesame 0x4ac3198 thru 0x4acf5e3 data_size 0x22(0xc44c bytes) 2024-09-21T22:50:45.806 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21054 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8257417280000001, 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055291275000000001, 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21054 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979167+0000", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979170+0000", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0699999999999998e-06 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979182+0000", 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1843e-05 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979189+0000", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-06 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034243+0000", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055053671999999998 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034349+0000", 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105739 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034458+0000", 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109008 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21055 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8257097199999999, 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055396608, 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21055 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979199+0000", 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979202+0000", 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.05e-06 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979223+0000", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1311999999999999e-05 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979232+0000", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4530000000000002e-06 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034495+0000", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055262807999999997 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034539+0000", 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4926000000000003e-05 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034595+0000", 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5418999999999999e-05 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21056 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.825665742, 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055614907999999998, 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21056 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979243+0000", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3e-07 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979247+0000", 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6200000000000001e-06 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979268+0000", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1226e-05 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979278+0000", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0212e-05 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034644+0000", 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055365960999999998 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034749+0000", 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104849 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034857+0000", 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010831 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.825618784, 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:50:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8255887830000002, 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:50:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8255610770000001, 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:50:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:50:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8255151270000001, 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:50:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8254749920000002, 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8254449930000001, 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:50:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:50:45.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.825410932, 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:50:45.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:50:45.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8253670770000001, 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:45.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8253309460000002, 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:50:45.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:50:45.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8252803650000002, 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:45.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:50:45.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:50:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8251751509999998, 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:50:45.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:50:45.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:50:45.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8251116409999999, 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:50:45.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:50:45.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8250134390000001, 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:50:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:50:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8249481759999999, 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8248822589999998, 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8248051570000001, 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:50:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:50:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8247315569999998, 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:50:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:45.891 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:45.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2775 read 0xb4de6ed thru 0xb4e8cce (0xa5e2 bytes) 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2778 read 0xcb51990 thru 0xcb5bd40 (0xa3b1 bytes) 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2779 write 0x74ce7b thru 0x74fe79 (0x2fff bytes) 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2780 writesame 0x7a52734 thru 0x7a5f6ae data_size 0x5(0xcf7b bytes) 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2781 read 0xc8ffc80 thru 0xc902b6f (0x2ef0 bytes) 2024-09-21T22:50:45.893 INFO:teuthology.orchestra.run.smithi094.stdout:2782 write 0x1fd9584 thru 0x1fddfb6 (0x4a33 bytes) 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:2784 write 0xcc5df12 thru 0xcc6bc97 (0xdd86 bytes) 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:2785 punch from 0xe7df12a to 0xe7eaf50, (0xbe26 bytes) 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:2786 read 0x1577ba8 thru 0x157a965 (0x2dbe bytes) 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:2788 read 0xa601c65 thru 0xa60e296 (0xc632 bytes) 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:45.894 INFO:teuthology.orchestra.run.smithi094.stdout:2790 write 0xae08415 thru 0xae15af5 (0xd6e1 bytes) 2024-09-21T22:50:45.894 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:45.884+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:45.895 INFO:teuthology.orchestra.run.smithi094.stdout:2791 read 0x96a609 thru 0x979c1d (0xf615 bytes) 2024-09-21T22:50:45.896 INFO:teuthology.orchestra.run.smithi094.stdout:2792 clone 14 order 21 su 32768 sc 2 2024-09-21T22:50:45.992 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:45.989+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:46.059 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:46.059 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:46.059 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:46.060 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:46.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:50:46.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:46.061 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T22:50:46.061 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T22:50:46.061 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T22:50:46.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T22:50:46.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:46.117 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:46.117+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:46.193 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:46.193+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:46.201 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T22:50:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone13 from 0xeb48e87 (overlap 0x133953f) to 0xd74edf 2024-09-21T22:50:46.263 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:46.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:46.263 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:46.263 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:46.263 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:46.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:46.294 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:46.288+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:46.395 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:46.393+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:46.456 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.060793143000001, 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.060749713, 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.060641121, 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9346 3.c 3:343543ac:::rbd_data.111e1fb363f5.0000000000000002:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.060249423999998, 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380256999999998, 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9346 2024-09-21T22:50:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392892+0000", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392893+0000", 2024-09-21T22:50:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392898+0000", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1429999999999997e-06 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392918+0000", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0165000000000001e-05 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392932+0000", 2024-09-21T22:50:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3073e-05 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.427888+0000", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034956787000000003 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428089+0000", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020047700000000001 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.428272+0000", 2024-09-21T22:50:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000183575 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.051279534999999, 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35808 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 759808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1601521939999999, 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035722402, 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35808 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292991+0000", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3769999999999999e-06 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292998+0000", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2359999999999998e-06 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293023+0000", 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4765000000000001e-05 2024-09-21T22:50:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293038+0000", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5257e-05 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328503+0000", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035465093000000003 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328609+0000", 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010592899999999999 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328712+0000", 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010274500000000001 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35809 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 804864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1600821539999999, 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035887960000000003, 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293061+0000", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293071+0000", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0530000000000001e-05 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293094+0000", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3224999999999999e-05 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293109+0000", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4646e-05 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328750+0000", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035641391000000001 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328851+0000", 2024-09-21T22:50:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026099999999999 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328947+0000", 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6617e-05 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35815 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.160013097, 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036042883999999997, 2024-09-21T22:50:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35815 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293130+0000", 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293135+0000", 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5600000000000001e-06 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293158+0000", 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2619000000000001e-05 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293172+0000", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4549000000000001e-05 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328988+0000", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035815391000000002 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329085+0000", 2024-09-21T22:50:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7112000000000001e-05 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329171+0000", 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6349999999999998e-05 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35816 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 796672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1599501400000001, 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036185001000000001, 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35816 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293193+0000", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2559999999999999e-06 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293199+0000", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293225+0000", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6213000000000001e-05 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293244+0000", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9338999999999999e-05 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329217+0000", 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035972956 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329300+0000", 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3010000000000007e-05 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329376+0000", 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6056999999999994e-05 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35817 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 841728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1598764100000001, 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036292586000000002, 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35817 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293266+0000", 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293289+0000", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2283e-05 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293335+0000", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6597000000000001e-05 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293351+0000", 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329406+0000", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036055023999999998 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329484+0000", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7039000000000006e-05 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329558+0000", 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4233999999999994e-05 2024-09-21T22:50:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35824 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.159764445, 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03636607, 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35824 2024-09-21T22:50:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293378+0000", 2024-09-21T22:50:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2359999999999999e-06 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293386+0000", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293408+0000", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2713000000000001e-05 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293424+0000", 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5502000000000002e-05 2024-09-21T22:50:46.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329588+0000", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036164302000000002 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329674+0000", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5447999999999999e-05 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329743+0000", 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9663e-05 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35825 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 833536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1596977150000001, 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036463540000000003, 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35825 2024-09-21T22:50:46.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293445+0000", 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1230000000000001e-06 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293452+0000", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4259999999999997e-06 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293478+0000", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5423e-05 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293493+0000", 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5262000000000001e-05 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329773+0000", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036279726999999998 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329847+0000", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3842000000000002e-05 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329907+0000", 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0736999999999998e-05 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35833 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1596034500000001, 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036522798000000002, 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35833 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293539+0000", 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-21T22:50:46.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293547+0000", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.14e-06 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293572+0000", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4638999999999999e-05 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293587+0000", 2024-09-21T22:50:46.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5362e-05 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329935+0000", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036347154999999999 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330007+0000", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2322000000000003e-05 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330061+0000", 2024-09-21T22:50:46.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3953000000000003e-05 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35840 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 561152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.159533733, 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036597440000000002, 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35840 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293609+0000", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.15e-06 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293615+0000", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0399999999999998e-06 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293708+0000", 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2972000000000004e-05 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293768+0000", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9744000000000003e-05 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330088+0000", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036320676000000003 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330135+0000", 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6113999999999999e-05 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330205+0000", 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0743999999999999e-05 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35842 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1593025240000001, 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036523119999999999, 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35842 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293841+0000", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.474e-06 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293849+0000", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6729999999999998e-06 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293871+0000", 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2291999999999999e-05 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293888+0000", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512999999999999e-05 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330234+0000", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036346373000000001 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330308+0000", 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3770999999999996e-05 2024-09-21T22:50:46.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330362+0000", 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4024000000000003e-05 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35848 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 552960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1592319899999999, 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036565919000000002, 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35848 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:46.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293911+0000", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1570000000000001e-06 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293916+0000", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294560+0000", 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00064339699999999998 2024-09-21T22:50:46.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294578+0000", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8318999999999999e-05 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330391+0000", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035812716000000001 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330434+0000", 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3421000000000003e-05 2024-09-21T22:50:46.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330475+0000", 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1242999999999998e-05 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35849 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 598016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1585325900000001, 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036033295999999999, 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35849 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294611+0000", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-06 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294620+0000", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1300000000000007e-06 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294643+0000", 2024-09-21T22:50:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2849000000000001e-05 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294660+0000", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7419e-05 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330504+0000", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035843815000000001 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330571+0000", 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7446000000000004e-05 2024-09-21T22:50:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330642+0000", 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0900000000000002e-05 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35850 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 643072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.158460783, 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036113026999999999, 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35850 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294682+0000", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2240000000000001e-06 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294688+0000", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0530000000000003e-06 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294725+0000", 2024-09-21T22:50:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807999999999997e-05 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295530+0000", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080478500000000003 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330672+0000", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035142709000000001 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330748+0000", 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5794e-05 2024-09-21T22:50:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330794+0000", 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5654e-05 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35857 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.157506414, 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035441063000000002, 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35857 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295636+0000", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295642+0000", 2024-09-21T22:50:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295662+0000", 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9775e-05 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295677+0000", 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4606e-05 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330937+0000", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035260540999999999 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330998+0000", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0763000000000001e-05 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331076+0000", 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8197999999999999e-05 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35858 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1574459070000001, 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035527577999999997, 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35858 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295697+0000", 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295703+0000", 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.993e-06 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295740+0000", 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6984999999999999e-05 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.296513+0000", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077355700000000002 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331106+0000", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034593169999999999 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331179+0000", 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2705000000000003e-05 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331223+0000", 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4091000000000002e-05 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T22:50:46.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:46.543 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:46.536+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:46.643 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:46.636+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:46.729 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:46.729 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:46.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:46.744 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:46.741+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:46.754 INFO:teuthology.orchestra.run.smithi094.stdout:2793 trunc from 0xeb48e87 to 0x6b6b738 2024-09-21T22:50:46.781 INFO:teuthology.orchestra.run.smithi094.stdout:2795 trunc from 0x6b6b738 to 0x95fa2e0 2024-09-21T22:50:46.803 INFO:teuthology.orchestra.run.smithi094.stdout:2796 trunc from 0x95fa2e0 to 0x1e9cefe 2024-09-21T22:50:46.845 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:46.840+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:46.872 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:46.873 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:46.886 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:46.946 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:46.945+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:47.009 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 54.071004922999997, 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52599744800000003, 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:50:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52593306399999995, 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52586961399999999, 2024-09-21T22:50:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52580643599999999, 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:50:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:50:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.525750096, 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:50:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52570835400000004, 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.525647753, 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52556731700000003, 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:50:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:50:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52549120699999996, 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:50:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52540413399999997, 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:50:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52536064599999999, 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:50:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52532700799999998, 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52529324200000005, 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52526195799999997, 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:50:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52521660299999995, 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:50:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:50:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52518510699999998, 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:50:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.525150326, 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.50720479500000004, 2024-09-21T22:50:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.50714375099999998, 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:50:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:47.095 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:47.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:47.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:50:47.097 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:47.088+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:47.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:47.161 INFO:teuthology.orchestra.run.smithi094.stdout:2797 write 0xa6582fc thru 0xa6615f5 (0x92fa bytes) 2024-09-21T22:50:47.168 INFO:teuthology.orchestra.run.smithi094.stdout:2798 writesame 0x612a48 thru 0x6174c0 data_size 0x5d(0x4a79 bytes) 2024-09-21T22:50:47.196 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:47.192+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:47.219 INFO:teuthology.orchestra.run.smithi094.stdout:2799 punch from 0x84189c3 to 0x84267fd, (0xde3a bytes) 2024-09-21T22:50:47.219 INFO:teuthology.orchestra.run.smithi094.stdout:2800 punch from 0x2d2b8e7 to 0x2d33cc0, (0x83d9 bytes) 2024-09-21T22:50:47.219 INFO:teuthology.orchestra.run.smithi094.stdout:2801 read 0x4128e26 thru 0x41295ef (0x7ca bytes) 2024-09-21T22:50:47.219 INFO:teuthology.orchestra.run.smithi094.stdout:2802 read 0x6167097 thru 0x617551f (0xe489 bytes) 2024-09-21T22:50:47.219 INFO:teuthology.orchestra.run.smithi094.stdout:2804 read 0x1b884d9 thru 0x1b8f0fc (0x6c24 bytes) 2024-09-21T22:50:47.234 INFO:teuthology.orchestra.run.smithi094.stdout:2805 write 0xe0c8a24 thru 0xe0cf2ed (0x68ca bytes) 2024-09-21T22:50:47.239 INFO:teuthology.orchestra.run.smithi094.stdout:2806 write 0xafb0415 thru 0xafb115f (0xd4b bytes) 2024-09-21T22:50:47.257 INFO:teuthology.orchestra.run.smithi094.stdout:2807 trunc from 0xe0cf2ee to 0x3d4074a 2024-09-21T22:50:47.260 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:47.273 INFO:teuthology.orchestra.run.smithi094.stdout:2808 trunc from 0x3d4074a to 0x49e2e37 2024-09-21T22:50:47.278 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:47.282 INFO:teuthology.orchestra.run.smithi094.stdout:2809 trunc from 0x49e2e37 to 0x3507740 2024-09-21T22:50:47.291 INFO:teuthology.orchestra.run.smithi094.stdout:2810 trunc from 0x3507740 to 0x27d651d 2024-09-21T22:50:47.292 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:47.292 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:47.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:47.297 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:47.292+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:47.298 INFO:teuthology.orchestra.run.smithi094.stdout:2811 read 0x1e543bf thru 0x1e57b00 (0x3742 bytes) 2024-09-21T22:50:47.299 INFO:teuthology.orchestra.run.smithi094.stdout:2812 read 0x47452a thru 0x47bc79 (0x7750 bytes) 2024-09-21T22:50:47.300 INFO:teuthology.orchestra.run.smithi094.stdout:2813 punch from 0x2423f58 to 0x242f4c1, (0xb569 bytes) 2024-09-21T22:50:47.300 INFO:teuthology.orchestra.run.smithi094.stdout:2815 read 0xf23141 thru 0xf28fff (0x5ebf bytes) 2024-09-21T22:50:47.301 INFO:teuthology.orchestra.run.smithi094.stdout:2816 write 0x1e4b5a7 thru 0x1e50444 (0x4e9e bytes) 2024-09-21T22:50:47.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:50:47.322 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:47.322 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:47.325 INFO:teuthology.orchestra.run.smithi094.stdout:2817 writesame 0xbaeb6b5 thru 0xbaece2f data_size 0x1(0x177b bytes) 2024-09-21T22:50:47.325 INFO:teuthology.orchestra.run.smithi094.stdout:2819 writesame 0x2905028 thru 0x290d312 data_size 0x1(0x82eb bytes) 2024-09-21T22:50:47.340 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:47.340 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:47.348 INFO:teuthology.orchestra.run.smithi094.stdout:2820 trunc from 0xbaece30 to 0xd7f116e 2024-09-21T22:50:47.356 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.356 INFO:teuthology.orchestra.run.smithi094.stdout:2824 read 0x1a84330 thru 0x1a8ee08 (0xaad9 bytes) 2024-09-21T22:50:47.356 INFO:teuthology.orchestra.run.smithi094.stdout:2825 punch from 0xc70472f to 0xc70b12e, (0x69ff bytes) 2024-09-21T22:50:47.356 INFO:teuthology.orchestra.run.smithi094.stdout:2826 punch from 0xacbcf3c to 0xacc35eb, (0x66af bytes) 2024-09-21T22:50:47.356 INFO:teuthology.orchestra.run.smithi094.stdout:2827 read 0xc1c6b47 thru 0xc1d4d5e (0xe218 bytes) 2024-09-21T22:50:47.357 INFO:teuthology.orchestra.run.smithi094.stdout:2828 trunc from 0xd7f116e to 0xd231ef5 2024-09-21T22:50:47.359 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:47.365 INFO:teuthology.orchestra.run.smithi094.stdout:2829 read 0x819a1ca thru 0x819c9fb (0x2832 bytes) 2024-09-21T22:50:47.365 INFO:teuthology.orchestra.run.smithi094.stdout:2830 read 0x543153f thru 0x543bad9 (0xa59b bytes) 2024-09-21T22:50:47.365 INFO:teuthology.orchestra.run.smithi094.stdout:2831 writesame 0xbaf53cf thru 0xbaf9f74 data_size 0x17(0x4ba6 bytes) 2024-09-21T22:50:47.370 INFO:teuthology.orchestra.run.smithi094.stdout:2833 read 0x5f2a714 thru 0x5f3815e (0xda4b bytes) 2024-09-21T22:50:47.370 INFO:teuthology.orchestra.run.smithi094.stdout:2834 read 0x69cb6a2 thru 0x69da4af (0xee0e bytes) 2024-09-21T22:50:47.370 INFO:teuthology.orchestra.run.smithi094.stdout:2835 write 0x4f6624b thru 0x4f6f1f4 (0x8faa bytes) 2024-09-21T22:50:47.374 INFO:teuthology.orchestra.run.smithi094.stdout:2836 write 0xc001102 thru 0xc00cde9 (0xbce8 bytes) 2024-09-21T22:50:47.378 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:47.378 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:47.379 INFO:teuthology.orchestra.run.smithi094.stdout:2837 read 0xc7f387a thru 0xc8031a7 (0xf92e bytes) 2024-09-21T22:50:47.379 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.381 INFO:teuthology.orchestra.run.smithi094.stdout:2841 trunc from 0xd231ef5 to 0xe7be554 2024-09-21T22:50:47.396 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:47.397 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:47.396+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:47.402 INFO:teuthology.orchestra.run.smithi094.stdout:2842 trunc from 0xe7be554 to 0x4b59e74 2024-09-21T22:50:47.427 INFO:teuthology.orchestra.run.smithi094.stdout:2843 write 0x50f6f36 thru 0x51033ed (0xc4b8 bytes) 2024-09-21T22:50:47.434 INFO:teuthology.orchestra.run.smithi094.stdout:2844 read 0x252f0a3 thru 0x253be6e (0xcdcc bytes) 2024-09-21T22:50:47.456 INFO:teuthology.orchestra.run.smithi094.stdout:2846 write 0xd79c7bb thru 0xd7a58e1 (0x9127 bytes) 2024-09-21T22:50:47.467 INFO:teuthology.orchestra.run.smithi094.stdout:2848 writesame 0x90cfe49 thru 0x90d4cb7 data_size 0x17(0x4e6f bytes) 2024-09-21T22:50:47.475 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.475 INFO:teuthology.orchestra.run.smithi094.stdout:2851 flatten 2024-09-21T22:50:47.484 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:47.484 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:47.498 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:47.492+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:47.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:50:47.602 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:47.600+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:47.699 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:47.696+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:47.716 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:47.717 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:47.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:47.771 INFO:teuthology.orchestra.run.smithi094.stdout:2853 write 0xca28c74 thru 0xca324b8 (0x9845 bytes) 2024-09-21T22:50:47.778 INFO:teuthology.orchestra.run.smithi094.stdout:2854 writesame 0xcdf04ed thru 0xcdffaea data_size 0x2e(0xf5fe bytes) 2024-09-21T22:50:47.783 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.784 INFO:teuthology.orchestra.run.smithi094.stdout:2856 write 0x7e57639 thru 0x7e57bd3 (0x59b bytes) 2024-09-21T22:50:47.787 INFO:teuthology.orchestra.run.smithi094.stdout:2857 read 0x22cbc31 thru 0x22d4043 (0x8413 bytes) 2024-09-21T22:50:47.787 INFO:teuthology.orchestra.run.smithi094.stdout:2858 read 0x45034c8 thru 0x4505cdf (0x2818 bytes) 2024-09-21T22:50:47.787 INFO:teuthology.orchestra.run.smithi094.stdout:2859 punch from 0x9682143 to 0x96884d9, (0x6396 bytes) 2024-09-21T22:50:47.787 INFO:teuthology.orchestra.run.smithi094.stdout:2860 read 0xe09a4c thru 0xe169a0 (0xcf55 bytes) 2024-09-21T22:50:47.788 INFO:teuthology.orchestra.run.smithi094.stdout:2861 read 0x37e6889 thru 0x37e8e94 (0x260c bytes) 2024-09-21T22:50:47.788 INFO:teuthology.orchestra.run.smithi094.stdout:2862 writesame 0xb9cc7f6 thru 0xb9d4e06 data_size 0x1(0x8611 bytes) 2024-09-21T22:50:47.800 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:47.792+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:47.800 INFO:teuthology.orchestra.run.smithi094.stdout:2863 write 0x63ec5c4 thru 0x63f794c (0xb389 bytes) 2024-09-21T22:50:47.810 INFO:teuthology.orchestra.run.smithi094.stdout:2864 writesame 0xbacbfdd thru 0xbadab70 data_size 0x2(0xeb94 bytes) 2024-09-21T22:50:47.840 INFO:teuthology.orchestra.run.smithi094.stdout:2865 trunc from 0xd7a58e2 to 0x1a87e37 2024-09-21T22:50:47.871 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:47.871 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:47.872 INFO:teuthology.orchestra.run.smithi094.stdout:2866 read 0x6d80ac thru 0x6dc878 (0x47cd bytes) 2024-09-21T22:50:47.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:47.894 INFO:teuthology.orchestra.run.smithi094.stdout:2868 writesame 0x6adf596 thru 0x6aeae68 data_size 0x1(0xb8d3 bytes) 2024-09-21T22:50:47.894 INFO:teuthology.orchestra.run.smithi094.stdout:2869 read 0x5029c84 thru 0x5036812 (0xcb8f bytes) 2024-09-21T22:50:47.901 INFO:teuthology.orchestra.run.smithi094.stdout:2870 trunc from 0x6aeae69 to 0xb3f7c07 2024-09-21T22:50:47.901 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:47.896+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:47.918 INFO:teuthology.orchestra.run.smithi094.stdout:2871 trunc from 0xb3f7c07 to 0x8642d3b 2024-09-21T22:50:47.932 INFO:teuthology.orchestra.run.smithi094.stdout:2872 read 0x7ef6428 thru 0x7efe72d (0x8306 bytes) 2024-09-21T22:50:47.952 INFO:teuthology.orchestra.run.smithi094.stdout:2874 write 0xc9e032d thru 0xc9e8988 (0x865c bytes) 2024-09-21T22:50:47.980 INFO:teuthology.orchestra.run.smithi094.stdout:2875 writesame 0xea177f5 thru 0xea1fea4 data_size 0x28(0x86b0 bytes) 2024-09-21T22:50:47.980 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.980 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2879 punch from 0xc56ea0d to 0xc5700cd, (0x16c0 bytes) 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2880 punch from 0xd204d8e to 0xd205143, (0x3b5 bytes) 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2881 read 0x4bbf6d2 thru 0x4bceea2 (0xf7d1 bytes) 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2882 read 0xdd07d9a thru 0xdd0bd4d (0x3fb4 bytes) 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2884 punch from 0xaac651f to 0xaad1e71, (0xb952 bytes) 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xed93c0d 2024-09-21T22:50:47.981 INFO:teuthology.orchestra.run.smithi094.stdout:2885 trunc from 0xea1fea5 to 0xed93c0d 2024-09-21T22:50:47.996 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.996 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:47.996 INFO:teuthology.orchestra.run.smithi094.stdout:2888 write 0x6602202 thru 0x6602c5c (0xa5b bytes) 2024-09-21T22:50:48.001 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:48.000+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:48.003 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:48.003 INFO:teuthology.orchestra.run.smithi094.stdout:2890 punch from 0xc36082c to 0xc36ff36, (0xf70a bytes) 2024-09-21T22:50:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:2891 write 0x31c4cb2 thru 0x31ca3d9 (0x5728 bytes) 2024-09-21T22:50:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:2892 punch from 0x8384433 to 0x8389825, (0x53f2 bytes) 2024-09-21T22:50:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:2894 read 0x9022acd thru 0x902efae (0xc4e2 bytes) 2024-09-21T22:50:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:2895 writesame 0x584c63f thru 0x585a77e data_size 0x40(0xe140 bytes) 2024-09-21T22:50:48.023 INFO:teuthology.orchestra.run.smithi094.stdout:2897 read 0x75dcb06 thru 0x75e2c13 (0x610e bytes) 2024-09-21T22:50:48.023 INFO:teuthology.orchestra.run.smithi094.stdout:2899 read 0xabae70b thru 0xabb72bc (0x8bb2 bytes) 2024-09-21T22:50:48.023 INFO:teuthology.orchestra.run.smithi094.stdout:2902 read 0x7ea5a9d thru 0x7ea78cf (0x1e33 bytes) 2024-09-21T22:50:48.031 INFO:teuthology.orchestra.run.smithi094.stdout:2903 trunc from 0xed93c0d to 0x9c77adc 2024-09-21T22:50:48.033 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.033 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:48.057 INFO:teuthology.orchestra.run.smithi094.stdout:2904 write 0x185582e thru 0x1861d7d (0xc550 bytes) 2024-09-21T22:50:48.068 INFO:teuthology.orchestra.run.smithi094.stdout:2905 punch from 0x467b1f5 to 0x467ee80, (0x3c8b bytes) 2024-09-21T22:50:48.072 INFO:teuthology.orchestra.run.smithi094.stdout:2906 trunc from 0x9c77adc to 0xc5305b1 2024-09-21T22:50:48.084 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:50:48.094 INFO:teuthology.orchestra.run.smithi094.stdout:2909 write 0x3495877 thru 0x349b2ae (0x5a38 bytes) 2024-09-21T22:50:48.102 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:48.096+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:48.109 INFO:teuthology.orchestra.run.smithi094.stdout:2910 trunc from 0xc5305b1 to 0xd48896f 2024-09-21T22:50:48.124 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:48.137 INFO:teuthology.orchestra.run.smithi094.stdout:2914 write 0xd843df4 thru 0xd848569 (0x4776 bytes) 2024-09-21T22:50:48.148 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:48.149 INFO:teuthology.orchestra.run.smithi094.stdout:2917 punch from 0xabe61c9 to 0xabf0ff2, (0xae29 bytes) 2024-09-21T22:50:48.149 INFO:teuthology.orchestra.run.smithi094.stdout:2919 writesame 0xd4ee749 thru 0xd4eefcf data_size 0x1(0x887 bytes) 2024-09-21T22:50:48.156 INFO:teuthology.orchestra.run.smithi094.stdout:2920 clone 15 order 24 su 8388608 sc 10 2024-09-21T22:50:48.186 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.186 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:48.203 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:48.200+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:48.304 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:48.296+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:48.362 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.362 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:48.421 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:48.416+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:48.506 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:48.500+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:48.543 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.543 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:48.607 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:48.604+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:48.708 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:48.704+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:48.722 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.722 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:48.809 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:48.804+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:48.856 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:48.856 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:48.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:48.910 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:48.908+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:49.011 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:49.008+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:49.037 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:49.037 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:49.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:49.112 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:49.104+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:49.171 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:49.171 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:49.187 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:49.212 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:49.208+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:49.313 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:49.308+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:49.356 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:49.356 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:49.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:49.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:49.421 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:49.416+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:49.470 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T22:50:49.514 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:49.512+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:49.582 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:49.582 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:49.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:49.603 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:49.615 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:49.612+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:49.620 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:49.638 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:49.654 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:49.655 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:49.675 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:49.675 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:49.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:49.696 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:49.701 INFO:teuthology.orchestra.run.smithi094.stdout:2921 trunc from 0xd84856a to 0x6f7a6dd 2024-09-21T22:50:49.716 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:49.708+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:49.722 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:49.722 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:49.729 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:49.729 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:49.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:49.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:49.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:6462 3.2 3:44ccee00:::rbd_data.111ebc806776.000000000000001a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.965578406999999, 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.08074365, 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 6462 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762172+0000", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762174+0000", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3029999999999998e-06 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762186+0000", 2024-09-21T22:50:49.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.136e-05 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762217+0000", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0663000000000001e-05 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762232+0000", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5099999999999999e-05 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762388+0000", 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000156362 2024-09-21T22:50:49.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.762632+0000", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243995 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.777316+0000", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014684262999999999 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842690+0000", 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065373938000000006 2024-09-21T22:50:49.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842722+0000", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2283999999999998e-05 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842825+0000", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102192 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:22.842916+0000", 2024-09-21T22:50:49.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1189999999999999e-05 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:13534 3.1e 3:7f270d40:::rbd_data.111e4c6c7d62.000000000000022a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.701093432, 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0094102410000001, 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13534 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026657+0000", 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026659+0000", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.544e-06 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026666+0000", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0679999999999999e-06 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026681+0000", 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4758e-05 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026720+0000", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9595999999999998e-05 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026822+0000", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101512 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:36.026833+0000", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1698999999999999e-05 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.031773+0000", 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049400070000001 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032053+0000", 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027928499999999999 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.032241+0000", 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000188448 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.034179+0000", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001938214 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035722+0000", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001542762 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.035951+0000", 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000229232 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.036067+0000", 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116116 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38251 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 954368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.530854+0000", 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1968966459999999, 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063692304000000005, 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38251 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530854+0000", 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530854+0000", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530855+0000", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.61e-07 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530858+0000", 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2820000000000001e-06 2024-09-21T22:50:49.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530869+0000", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1112e-05 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530878+0000", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8890000000000008e-06 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594431+0000", 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063552771999999993 2024-09-21T22:50:49.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594501+0000", 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0278999999999996e-05 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594546+0000", 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5209e-05 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38253 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 978944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.530888+0000", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1968630820000001, 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063820554000000002, 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38253 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530888+0000", 2024-09-21T22:50:49.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530888+0000", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530888+0000", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7700000000000004e-07 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530891+0000", 2024-09-21T22:50:49.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9189999999999999e-06 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530903+0000", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1514e-05 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530909+0000", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8279999999999997e-06 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594563+0000", 2024-09-21T22:50:49.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063653232000000004 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594638+0000", 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5129999999999994e-05 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594708+0000", 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0353999999999998e-05 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38255 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 989184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.530919+0000", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1968320719999999, 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063946033999999999, 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38255 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530919+0000", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530919+0000", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530919+0000", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9400000000000005e-07 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530922+0000", 2024-09-21T22:50:49.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.74e-06 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530933+0000", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1001e-05 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530950+0000", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7322000000000001e-05 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594725+0000", 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063774616000000006 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594801+0000", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6687000000000006e-05 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594865+0000", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3074e-05 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38257 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 999424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.530961+0000", 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1967896279999999, 2024-09-21T22:50:49.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064072469000000007, 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38257 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530961+0000", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530961+0000", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530962+0000", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2500000000000005e-07 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530965+0000", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530976+0000", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1538e-05 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530983+0000", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9800000000000001e-06 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594881+0000", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063898065000000004 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.594963+0000", 2024-09-21T22:50:49.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1607999999999995e-05 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595033+0000", 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0525000000000007e-05 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38259 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 1013760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.530992+0000", 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1967583930000001, 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064211104000000005, 2024-09-21T22:50:49.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38259 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530992+0000", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530992+0000", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530993+0000", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0900000000000001e-07 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.530996+0000", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.757e-06 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531007+0000", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1144e-05 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531013+0000", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2210000000000002e-06 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595059+0000", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064046497999999993 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595128+0000", 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8905999999999994e-05 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595203+0000", 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4969000000000007e-05 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38261 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 1024000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.531024+0000", 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1967268959999999, 2024-09-21T22:50:49.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064369329000000003, 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38261 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531024+0000", 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531024+0000", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531024+0000", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9299999999999997e-07 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531028+0000", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2789999999999999e-06 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531039+0000", 2024-09-21T22:50:49.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0847e-05 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531045+0000", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2210000000000002e-06 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595242+0000", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064197330999999996 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595321+0000", 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8811000000000001e-05 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595393+0000", 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2146999999999999e-05 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38263 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 1034240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.531054+0000", 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1966970030000001, 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064497589999999994, 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38263 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531054+0000", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531054+0000", 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531054+0000", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9999999999999997e-07 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531073+0000", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8924999999999999e-05 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531083+0000", 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0149999999999999e-05 2024-09-21T22:50:49.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531090+0000", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7150000000000001e-06 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595436+0000", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064346176000000005 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595501+0000", 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4874000000000003e-05 2024-09-21T22:50:49.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595551+0000", 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0149999999999999e-05 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38296 3.8 3:14c6bf0b:::rbd_header.111e38708da6:head [watch reconnect cookie 140232629978448 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.531100+0000", 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1966506479999999, 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065885506999999996, 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38296 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531100+0000", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531100+0000", 2024-09-21T22:50:49.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531101+0000", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531104+0000", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.196e-06 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531116+0000", 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1738e-05 2024-09-21T22:50:49.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531122+0000", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8340000000000001e-06 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595423+0000", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064300863 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.595483+0000", 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0139000000000001e-05 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.596985+0000", 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001502001 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38454 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 718848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533360+0000", 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1943906210000002, 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063772111000000006, 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38454 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533360+0000", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533360+0000", 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533361+0000", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5599999999999995e-07 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533363+0000", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9529999999999999e-06 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533373+0000", 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9599999999999995e-06 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533381+0000", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1250000000000004e-06 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.596961+0000", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063580167000000007 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597051+0000", 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9816999999999997e-05 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597132+0000", 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1532999999999993e-05 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38455 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 729088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533390+0000", 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.194360664, 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063941200000000004, 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38455 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533390+0000", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533390+0000", 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533391+0000", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0900000000000001e-07 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533394+0000", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0939999999999999e-06 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533402+0000", 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1130000000000001e-06 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533408+0000", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0220000000000001e-06 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597166+0000", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063758340999999996 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597253+0000", 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6876e-05 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597331+0000", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8145000000000005e-05 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38456 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 739328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533417+0000", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1943337810000001, 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064145324000000004, 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38456 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533417+0000", 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533417+0000", 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533417+0000", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.68e-07 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533421+0000", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7560000000000001e-06 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533429+0000", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8339999999999999e-06 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533435+0000", 2024-09-21T22:50:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5969999999999999e-06 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597377+0000", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063942299999999994 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597477+0000", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9815000000000006e-05 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597562+0000", 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5453999999999999e-05 2024-09-21T22:50:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38457 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 763904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533507+0000", 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.194243218, 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064264257000000005, 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38457 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533507+0000", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533507+0000", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533508+0000", 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533513+0000", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6190000000000002e-06 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533524+0000", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.11e-05 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533532+0000", 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7239999999999992e-06 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597602+0000", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064070515999999994 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597690+0000", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7594000000000002e-05 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597772+0000", 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1914000000000002e-05 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38458 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 774144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533544+0000", 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1942069260000001, 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064430771999999997, 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38458 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533544+0000", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533544+0000", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533545+0000", 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7400000000000002e-07 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533554+0000", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2280000000000004e-06 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533563+0000", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8720000000000008e-06 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533569+0000", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4479999999999999e-06 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597817+0000", 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064248100000000002 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597901+0000", 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.407e-05 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597974+0000", 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3180000000000001e-05 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38459 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 798720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533578+0000", 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1941724269999998, 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.06454348, 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38459 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533578+0000", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533578+0000", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533579+0000", 2024-09-21T22:50:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533581+0000", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6800000000000002e-06 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533589+0000", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533597+0000", 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0489999999999998e-06 2024-09-21T22:50:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.597992+0000", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064395419999999995 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598067+0000", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4919999999999994e-05 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598122+0000", 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4766999999999999e-05 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38460 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 808960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533606+0000", 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1941445740000001, 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064751569999999994, 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38460 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533606+0000", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533606+0000", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533607+0000", 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8400000000000004e-07 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533609+0000", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8700000000000001e-06 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533617+0000", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8769999999999993e-06 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533623+0000", 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9869999999999996e-06 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598170+0000", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064547058000000004 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598264+0000", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3468999999999996e-05 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598358+0000", 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3725000000000001e-05 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38461 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533631+0000", 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.194119309, 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064883904000000006, 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38461 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533631+0000", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533631+0000", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533632+0000", 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3799999999999997e-07 2024-09-21T22:50:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533634+0000", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5320000000000001e-06 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533643+0000", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3250000000000008e-06 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533649+0000", 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1279999999999996e-06 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598402+0000", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064752879999999999 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598468+0000", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6469000000000003e-05 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598515+0000", 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7032e-05 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38462 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 833536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533657+0000", 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1940931699999999, 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065004711000000007, 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38462 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533657+0000", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533657+0000", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533829+0000", 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017156699999999999 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533834+0000", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2329999999999997e-06 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533846+0000", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1997000000000001e-05 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533857+0000", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0739e-05 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598548+0000", 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064690580999999997 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598618+0000", 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0604000000000004e-05 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598662+0000", 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3989999999999997e-05 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:38464 3.18 3:1a18286b:::rbd_data.111ebb10c685.0000000000000002:e [read 843776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.533892+0000", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1938586920000001, 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064950372000000006, 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 38464 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533892+0000", 2024-09-21T22:50:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533892+0000", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536428+0000", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025363819999999998 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536435+0000", 2024-09-21T22:50:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1840000000000002e-06 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536451+0000", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5370999999999999e-05 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536460+0000", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5049999999999996e-06 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598704+0000", 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062243410999999998 2024-09-21T22:50:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598788+0000", 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4004000000000004e-05 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.598842+0000", 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4515000000000002e-05 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:49.812 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:49.813 INFO:teuthology.orchestra.run.smithi094.stdout:2923 read 0x4219b3 thru 0x42d6c9 (0xbd17 bytes) 2024-09-21T22:50:49.813 INFO:teuthology.orchestra.run.smithi094.stdout:2924 trunc from 0x6f7a6dd to 0x8589ea0 2024-09-21T22:50:49.813 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:49.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:49.815 INFO:teuthology.orchestra.run.smithi094.stdout:2926 write 0xb5cada8 thru 0xb5cd0f6 (0x234f bytes) 2024-09-21T22:50:49.815 INFO:teuthology.orchestra.run.smithi094.stdout:2927 read 0xb31c48c thru 0xb327cab (0xb820 bytes) 2024-09-21T22:50:49.815 INFO:teuthology.orchestra.run.smithi094.stdout:2928 write 0x7e3cd5c thru 0x7e46f94 (0xa239 bytes) 2024-09-21T22:50:49.816 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.816 INFO:teuthology.orchestra.run.smithi094.stdout:2930 read 0x268b218 thru 0x268f94b (0x4734 bytes) 2024-09-21T22:50:49.816 INFO:teuthology.orchestra.run.smithi094.stdout:2934 write 0x2923a0d thru 0x2925324 (0x1918 bytes) 2024-09-21T22:50:49.816 INFO:teuthology.orchestra.run.smithi094.stdout:2935 write 0x4db1e9c thru 0x4db5cf8 (0x3e5d bytes) 2024-09-21T22:50:49.816 INFO:teuthology.orchestra.run.smithi094.stdout:2936 trunc from 0xb5cd0f7 to 0x2a77a45 2024-09-21T22:50:49.856 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:49.852+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:49.876 INFO:teuthology.orchestra.run.smithi094.stdout:2938 read 0x1634b5a thru 0x163591f (0xdc6 bytes) 2024-09-21T22:50:49.877 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.877 INFO:teuthology.orchestra.run.smithi094.stdout:2940 read 0x243ab31 thru 0x2440a8d (0x5f5d bytes) 2024-09-21T22:50:49.896 INFO:teuthology.orchestra.run.smithi094.stdout:2941 write 0xa8c9b7f thru 0xa8cd746 (0x3bc8 bytes) 2024-09-21T22:50:49.902 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.903 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.914 INFO:teuthology.orchestra.run.smithi094.stdout:2947 write 0xbf4c260 thru 0xbf52996 (0x6737 bytes) 2024-09-21T22:50:49.917 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:49.916+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:49.921 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.934 INFO:teuthology.orchestra.run.smithi094.stdout:2950 write 0xd69f4a6 thru 0xd6ac904 (0xd45f bytes) 2024-09-21T22:50:49.940 INFO:teuthology.orchestra.run.smithi094.stdout:2951 writesame 0x8990515 thru 0x8995c03 data_size 0x1(0x56ef bytes) 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:49497 3.1a 3:5acbf86e:::rbd_data.111e5668b648.0000000000000007:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:49.944153+0000", 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0041659740000000002, 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042000739999999998, 2024-09-21T22:50:49.948 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 49497 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944153+0000", 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944153+0000", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944155+0000", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.654e-06 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944163+0000", 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1769999999999998e-06 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944180+0000", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7265000000000002e-05 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944188+0000", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8250000000000005e-06 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944240+0000", 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2042000000000001e-05 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.944298+0000", 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7890000000000003e-05 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.946587+0000", 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022884680000000001 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-21T22:50:49.952 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:49.957 INFO:teuthology.orchestra.run.smithi094.stdout:2953 writesame 0x709ac99 thru 0x70a33ac data_size 0x4(0x8714 bytes) 2024-09-21T22:50:49.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:49.965 INFO:teuthology.orchestra.run.smithi094.stdout:2956 read 0x14d6a25 thru 0x14e3fec (0xd5c8 bytes) 2024-09-21T22:50:49.966 INFO:teuthology.orchestra.run.smithi094.stdout:2958 read 0x99e63e0 thru 0x99e8cd8 (0x28f9 bytes) 2024-09-21T22:50:49.966 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.966 INFO:teuthology.orchestra.run.smithi094.stdout:2960 punch from 0x846bfa0 to 0x847bdc3, (0xfe23 bytes) 2024-09-21T22:50:49.966 INFO:teuthology.orchestra.run.smithi094.stdout:2961 write 0x55ea8a6 thru 0x55ed756 (0x2eb1 bytes) 2024-09-21T22:50:49.969 INFO:teuthology.orchestra.run.smithi094.stdout:2962 read 0x68c5f88 thru 0x68d22a4 (0xc31d bytes) 2024-09-21T22:50:49.970 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.970 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:49.970 INFO:teuthology.orchestra.run.smithi094.stdout:2969 write 0x179f8dc thru 0x17a0de9 (0x150e bytes) 2024-09-21T22:50:49.973 INFO:teuthology.orchestra.run.smithi094.stdout:2970 read 0x49862bb thru 0x49884fb (0x2241 bytes) 2024-09-21T22:50:49.973 INFO:teuthology.orchestra.run.smithi094.stdout:2971 write 0xd4f7bae thru 0xd500500 (0x8953 bytes) 2024-09-21T22:50:49.975 INFO:teuthology.orchestra.run.smithi094.stdout:2972 read 0x148dd2f thru 0x14965bb (0x888d bytes) 2024-09-21T22:50:49.977 INFO:teuthology.orchestra.run.smithi094.stdout:2973 writesame 0xbeeb671 thru 0xbefb631 data_size 0x1(0xffc1 bytes) 2024-09-21T22:50:50.019 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:50.012+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:50.049 INFO:teuthology.orchestra.run.smithi094.stdout:2974 punch from 0xa03c85a to 0xa03ca15, (0x1bb bytes) 2024-09-21T22:50:50.050 INFO:teuthology.orchestra.run.smithi094.stdout:2975 read 0x29b6eae thru 0x29b7110 (0x263 bytes) 2024-09-21T22:50:50.050 INFO:teuthology.orchestra.run.smithi094.stdout:2976 read 0xb5d3a02 thru 0xb5dfcbe (0xc2bd bytes) 2024-09-21T22:50:50.050 INFO:teuthology.orchestra.run.smithi094.stdout:2979 write 0xd531c24 thru 0xd532c55 (0x1032 bytes) 2024-09-21T22:50:50.053 INFO:teuthology.orchestra.run.smithi094.stdout:2980 writesame 0x4da4852 thru 0x4da8847 data_size 0x3(0x3ff6 bytes) 2024-09-21T22:50:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:2981 read 0x90f6098 thru 0x90fc42c (0x6395 bytes) 2024-09-21T22:50:50.084 INFO:teuthology.orchestra.run.smithi094.stdout:2982 trunc from 0xd6ac905 to 0x67a2939 2024-09-21T22:50:50.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:50:50.119 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:50.116+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:50.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:50.131 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:50.132 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:50.132 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:50.132 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:50.137 INFO:teuthology.orchestra.run.smithi094.stdout:2983 write 0xdef00df thru 0xdef2f1f (0x2e41 bytes) 2024-09-21T22:50:50.142 INFO:teuthology.orchestra.run.smithi094.stdout:2984 write 0xb8108a7 thru 0xb818a83 (0x81dd bytes) 2024-09-21T22:50:50.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:50.148 INFO:teuthology.orchestra.run.smithi094.stdout:2985 writesame 0x5dca9b4 thru 0x5dd6952 data_size 0x1(0xbf9f bytes) 2024-09-21T22:50:50.181 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.182 INFO:teuthology.orchestra.run.smithi094.stdout:2988 read 0xabbb0f thru 0xac1ea7 (0x6399 bytes) 2024-09-21T22:50:50.182 INFO:teuthology.orchestra.run.smithi094.stdout:2989 punch from 0x87fb5d4 to 0x8808efa, (0xd926 bytes) 2024-09-21T22:50:50.182 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.182 INFO:teuthology.orchestra.run.smithi094.stdout:2991 read 0x686027 thru 0x687905 (0x18df bytes) 2024-09-21T22:50:50.183 INFO:teuthology.orchestra.run.smithi094.stdout:2992 write 0xa015f85 thru 0xa01ae64 (0x4ee0 bytes) 2024-09-21T22:50:50.189 INFO:teuthology.orchestra.run.smithi094.stdout:2993 read 0x1deaf6d thru 0x1decca6 (0x1d3a bytes) 2024-09-21T22:50:50.208 INFO:teuthology.orchestra.run.smithi094.stdout:2994 trunc from 0xdef2f20 to 0x41c1cc0 2024-09-21T22:50:50.220 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:50.212+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:50.231 INFO:teuthology.orchestra.run.smithi094.stdout:2995 read 0x2bfb403 thru 0x2c03bb5 (0x87b3 bytes) 2024-09-21T22:50:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:2996 read 0x24c3317 thru 0x24cb13b (0x7e25 bytes) 2024-09-21T22:50:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:2997 write 0x2fc2bda thru 0x2fca66a (0x7a91 bytes) 2024-09-21T22:50:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:2998 writesame 0x16c9953 thru 0x16d9613 data_size 0x1(0xfcc1 bytes) 2024-09-21T22:50:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:2999 trunc from 0x41c1cc0 to 0x1549c4a 2024-09-21T22:50:50.296 INFO:teuthology.orchestra.run.smithi094.stdout:3000 read 0xd7a3dc thru 0xd82790 (0x83b5 bytes) 2024-09-21T22:50:50.297 INFO:teuthology.orchestra.run.smithi094.stdout:3001 trunc from 0x1549c4a to 0x203a8af 2024-09-21T22:50:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.305 INFO:teuthology.orchestra.run.smithi094.stdout:3003 read 0x60ab54 thru 0x619f69 (0xf416 bytes) 2024-09-21T22:50:50.306 INFO:teuthology.orchestra.run.smithi094.stdout:3004 write 0x190157a thru 0x190a257 (0x8cde bytes) 2024-09-21T22:50:50.310 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.310 INFO:teuthology.orchestra.run.smithi094.stdout:3007 read 0x1237a96 thru 0x123eecb (0x7436 bytes) 2024-09-21T22:50:50.321 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:50.316+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:50.327 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:50.327 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:50.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:50.327 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:50.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21054 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3460486300000003, 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055291275000000001, 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21054 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979166+0000", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979167+0000", 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979170+0000", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0699999999999998e-06 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979182+0000", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1843e-05 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979189+0000", 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-06 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034243+0000", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055053671999999998 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034349+0000", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105739 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034458+0000", 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109008 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21055 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3460166220000005, 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055396608, 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21055 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979198+0000", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979199+0000", 2024-09-21T22:50:50.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979202+0000", 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.05e-06 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979223+0000", 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1311999999999999e-05 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979232+0000", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4530000000000002e-06 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034495+0000", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055262807999999997 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034539+0000", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4926000000000003e-05 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034595+0000", 2024-09-21T22:50:50.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5418999999999999e-05 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21056 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3459726439999997, 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055614907999999998, 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21056 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979243+0000", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3e-07 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979247+0000", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6200000000000001e-06 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979268+0000", 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1226e-05 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979278+0000", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0212e-05 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034644+0000", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055365960999999998 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034749+0000", 2024-09-21T22:50:50.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104849 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034857+0000", 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010831 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3459256859999993, 2024-09-21T22:50:50.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:50:50.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:50:50.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.344 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3458956850000003, 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:50:50.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:50:50.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3458679789999994, 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:50:50.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3458220290000007, 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:50:50.353 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3457818939999999, 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:50:50.357 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3457518949999994, 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:50:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:50:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3457178340000002, 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:50:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3456739790000007, 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:50:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:50:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:50:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:50:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3456378480000009, 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:50:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:50:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3455872669999991, 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:50:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3454820529999996, 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:50:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:50:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:50:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:50:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3454185429999992, 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:50:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:50:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:50:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3453203410000008, 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:50:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3452550779999992, 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:50:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3451891610000004, 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3451120589999999, 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:50:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.3450384589999995, 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.410 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:50.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:3008 write 0x8113b16 thru 0x8119411 (0x58fc bytes) 2024-09-21T22:50:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:3009 trunc from 0x8119412 to 0xd13906b 2024-09-21T22:50:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:3013 read 0x2f11df9 thru 0x2f17c4f (0x5e57 bytes) 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:3014 read 0x53e0d7f thru 0x53e4445 (0x36c7 bytes) 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:3018 write 0xd9f6db4 thru 0xda061cc (0xf419 bytes) 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:3021 trunc from 0xda061cd to 0x686e169 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:3027 trunc from 0x686e169 to 0x5ba4023 2024-09-21T22:50:50.413 INFO:teuthology.orchestra.run.smithi094.stdout:3028 trunc from 0x5ba4023 to 0x65595d 2024-09-21T22:50:50.423 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:50.416+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:50.454 INFO:teuthology.orchestra.run.smithi094.stdout:3029 trunc from 0x65595d to 0xc370ccb 2024-09-21T22:50:50.462 INFO:teuthology.orchestra.run.smithi094.stdout:3031 read 0x1176293 thru 0x1183bc8 (0xd936 bytes) 2024-09-21T22:50:50.462 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.462 INFO:teuthology.orchestra.run.smithi094.stdout:3033 punch from 0x1273e3d to 0x12784c2, (0x4685 bytes) 2024-09-21T22:50:50.474 INFO:teuthology.orchestra.run.smithi094.stdout:3034 write 0xe514221 thru 0xe5196f3 (0x54d3 bytes) 2024-09-21T22:50:50.479 INFO:teuthology.orchestra.run.smithi094.stdout:3035 punch from 0x25901f4 to 0x259f7d7, (0xf5e3 bytes) 2024-09-21T22:50:50.495 INFO:teuthology.orchestra.run.smithi094.stdout:3036 trunc from 0xe5196f4 to 0x316f22a 2024-09-21T22:50:50.516 INFO:teuthology.orchestra.run.smithi094.stdout:3037 punch from 0x8f62b9 to 0x8f6b5a, (0x8a1 bytes) 2024-09-21T22:50:50.524 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:50.516+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:50.535 INFO:teuthology.orchestra.run.smithi094.stdout:3038 writesame 0x858dc32 thru 0x85998c0 data_size 0x1(0xbc8f bytes) 2024-09-21T22:50:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:3040 write 0xeff373 thru 0xf029d3 (0x3661 bytes) 2024-09-21T22:50:50.540 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:50.540 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:50.540 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:50.540 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:50.542 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:50.556 INFO:teuthology.orchestra.run.smithi094.stdout:3044 write 0xa56794b thru 0xa56a4b3 (0x2b69 bytes) 2024-09-21T22:50:50.562 INFO:teuthology.orchestra.run.smithi094.stdout:3045 read 0x31332b0 thru 0x313ee46 (0xbb97 bytes) 2024-09-21T22:50:50.562 INFO:teuthology.orchestra.run.smithi094.stdout:3047 write 0x1b02c48 thru 0x1b0b70d (0x8ac6 bytes) 2024-09-21T22:50:50.567 INFO:teuthology.orchestra.run.smithi094.stdout:3048 writesame 0x678ccec thru 0x67982a8 data_size 0x5(0xb5bd bytes) 2024-09-21T22:50:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:3049 write 0x2359de thru 0x23962c (0x3c4f bytes) 2024-09-21T22:50:50.578 INFO:teuthology.orchestra.run.smithi094.stdout:3051 read 0x1e8fbde thru 0x1e9ed3f (0xf162 bytes) 2024-09-21T22:50:50.593 INFO:teuthology.orchestra.run.smithi094.stdout:3052 trunc from 0xa56a4b4 to 0x331fa4b 2024-09-21T22:50:50.612 INFO:teuthology.orchestra.run.smithi094.stdout:3055 punch from 0x13d8f3e to 0x13e34e0, (0xa5a2 bytes) 2024-09-21T22:50:50.625 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:50.620+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:50.634 INFO:teuthology.orchestra.run.smithi094.stdout:3056 write 0x7b1d49f thru 0x7b26e3b (0x999d bytes) 2024-09-21T22:50:50.641 INFO:teuthology.orchestra.run.smithi094.stdout:3057 read 0x3d836a2 thru 0x3d8577d (0x20dc bytes) 2024-09-21T22:50:50.646 INFO:teuthology.orchestra.run.smithi094.stdout:3058 trunc from 0x7b26e3c to 0x9854304 2024-09-21T22:50:50.657 INFO:teuthology.orchestra.run.smithi094.stdout:3059 read 0x6879959 thru 0x6880035 (0x66dd bytes) 2024-09-21T22:50:50.658 INFO:teuthology.orchestra.run.smithi094.stdout:3061 write 0x1226a8a thru 0x1227810 (0xd87 bytes) 2024-09-21T22:50:50.661 INFO:teuthology.orchestra.run.smithi094.stdout:3062 write 0x5ef83 thru 0x68054 (0x90d2 bytes) 2024-09-21T22:50:50.665 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.665 INFO:teuthology.orchestra.run.smithi094.stdout:3064 read 0x7e97778 thru 0x7e98c1a (0x14a3 bytes) 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi094.stdout:3065 write 0xa6580cb thru 0xa658b00 (0xa36 bytes) 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:50.683 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:50.691 INFO:teuthology.orchestra.run.smithi094.stdout:3067 read 0x4c272e2 thru 0x4c317cb (0xa4ea bytes) 2024-09-21T22:50:50.691 INFO:teuthology.orchestra.run.smithi094.stdout:3068 read 0x74fd49f thru 0x750019c (0x2cfe bytes) 2024-09-21T22:50:50.691 INFO:teuthology.orchestra.run.smithi094.stdout:3070 punch from 0x121f551 to 0x1229fee, (0xaa9d bytes) 2024-09-21T22:50:50.691 INFO:teuthology.orchestra.run.smithi094.stdout:3071 write 0x447c482 thru 0x4480781 (0x4300 bytes) 2024-09-21T22:50:50.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:50.706 INFO:teuthology.orchestra.run.smithi094.stdout:3072 trunc from 0xa658b01 to 0xd5d26b9 2024-09-21T22:50:50.720 INFO:teuthology.orchestra.run.smithi094.stdout:3073 writesame 0x33e64b5 thru 0x33ec4ce data_size 0x1(0x601a bytes) 2024-09-21T22:50:50.726 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:50.720+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:50.741 INFO:teuthology.orchestra.run.smithi094.stdout:3074 punch from 0x317a421 to 0x31887fa, (0xe3d9 bytes) 2024-09-21T22:50:50.741 INFO:teuthology.orchestra.run.smithi094.stdout:3075 punch from 0x98a12a8 to 0x98af9db, (0xe733 bytes) 2024-09-21T22:50:50.743 INFO:teuthology.orchestra.run.smithi094.stdout:3077 trunc from 0xd5d26b9 to 0xe1d83e4 2024-09-21T22:50:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:3078 punch from 0x734dec7 to 0x7359d72, (0xbeab bytes) 2024-09-21T22:50:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:3079 write 0x7d9f7a0 thru 0x7da7685 (0x7ee6 bytes) 2024-09-21T22:50:50.760 INFO:teuthology.orchestra.run.smithi094.stdout:3080 write 0xcef1e99 thru 0xcefbaba (0x9c22 bytes) 2024-09-21T22:50:50.767 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.767 INFO:teuthology.orchestra.run.smithi094.stdout:3082 write 0x2c78ee1 thru 0x2c8725e (0xe37e bytes) 2024-09-21T22:50:50.803 INFO:teuthology.orchestra.run.smithi094.stdout:3083 trunc from 0xe1d83e4 to 0x2768fbc 2024-09-21T22:50:50.823 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.428071228, 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.428027797999999, 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:50.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.427919205999999, 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:50.832 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:50.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.418557620000001, 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:50.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35808 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 759808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5274302789999998, 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035722402, 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35808 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292991+0000", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3769999999999999e-06 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292998+0000", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2359999999999998e-06 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293023+0000", 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4765000000000001e-05 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293038+0000", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5257e-05 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328503+0000", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035465093000000003 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328609+0000", 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010592899999999999 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328712+0000", 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010274500000000001 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35809 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 804864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5273602390000001, 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035887960000000003, 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293061+0000", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293071+0000", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0530000000000001e-05 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293094+0000", 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3224999999999999e-05 2024-09-21T22:50:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293109+0000", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4646e-05 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328750+0000", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035641391000000001 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328851+0000", 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026099999999999 2024-09-21T22:50:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328947+0000", 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6617e-05 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35815 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5272911819999999, 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036042883999999997, 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35815 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293130+0000", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293135+0000", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5600000000000001e-06 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293158+0000", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2619000000000001e-05 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293172+0000", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4549000000000001e-05 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328988+0000", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035815391000000002 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329085+0000", 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7112000000000001e-05 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329171+0000", 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6349999999999998e-05 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35816 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 796672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.527228225, 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036185001000000001, 2024-09-21T22:50:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35816 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293193+0000", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2559999999999999e-06 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293199+0000", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293225+0000", 2024-09-21T22:50:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6213000000000001e-05 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293244+0000", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9338999999999999e-05 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329217+0000", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035972956 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329300+0000", 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3010000000000007e-05 2024-09-21T22:50:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329376+0000", 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6056999999999994e-05 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35817 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 841728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5271544949999996, 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036292586000000002, 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35817 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293266+0000", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293289+0000", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2283e-05 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293335+0000", 2024-09-21T22:50:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6597000000000001e-05 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293351+0000", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329406+0000", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036055023999999998 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329484+0000", 2024-09-21T22:50:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7039000000000006e-05 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329558+0000", 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4233999999999994e-05 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35824 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5270425300000001, 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03636607, 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35824 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293378+0000", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2359999999999999e-06 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293386+0000", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293408+0000", 2024-09-21T22:50:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2713000000000001e-05 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293424+0000", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5502000000000002e-05 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329588+0000", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036164302000000002 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329674+0000", 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5447999999999999e-05 2024-09-21T22:50:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329743+0000", 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9663e-05 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35825 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 833536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5269757999999998, 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036463540000000003, 2024-09-21T22:50:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35825 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293445+0000", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1230000000000001e-06 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293452+0000", 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4259999999999997e-06 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293478+0000", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5423e-05 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293493+0000", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5262000000000001e-05 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329773+0000", 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036279726999999998 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329847+0000", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3842000000000002e-05 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329907+0000", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0736999999999998e-05 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35833 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5268815350000002, 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036522798000000002, 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35833 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293539+0000", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293547+0000", 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.14e-06 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293572+0000", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4638999999999999e-05 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293587+0000", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5362e-05 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329935+0000", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036347154999999999 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330007+0000", 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2322000000000003e-05 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330061+0000", 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3953000000000003e-05 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35840 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 561152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5268118179999997, 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036597440000000002, 2024-09-21T22:50:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35840 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293609+0000", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.15e-06 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293615+0000", 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0399999999999998e-06 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293708+0000", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2972000000000004e-05 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293768+0000", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9744000000000003e-05 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330088+0000", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036320676000000003 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330135+0000", 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6113999999999999e-05 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330205+0000", 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0743999999999999e-05 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35842 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5265806089999998, 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036523119999999999, 2024-09-21T22:50:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35842 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293841+0000", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.474e-06 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293849+0000", 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6729999999999998e-06 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293871+0000", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2291999999999999e-05 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293888+0000", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512999999999999e-05 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330234+0000", 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036346373000000001 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330308+0000", 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3770999999999996e-05 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330362+0000", 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4024000000000003e-05 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35848 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 552960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.526510075, 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036565919000000002, 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35848 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293911+0000", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1570000000000001e-06 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293916+0000", 2024-09-21T22:50:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294560+0000", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00064339699999999998 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294578+0000", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8318999999999999e-05 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330391+0000", 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035812716000000001 2024-09-21T22:50:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330434+0000", 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3421000000000003e-05 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330475+0000", 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1242999999999998e-05 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35849 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 598016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5258106749999998, 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036033295999999999, 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35849 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294611+0000", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-06 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294620+0000", 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1300000000000007e-06 2024-09-21T22:50:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294643+0000", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2849000000000001e-05 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294660+0000", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7419e-05 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330504+0000", 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035843815000000001 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330571+0000", 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7446000000000004e-05 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330642+0000", 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0900000000000002e-05 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35850 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 643072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5257388680000004, 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036113026999999999, 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35850 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294682+0000", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2240000000000001e-06 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294688+0000", 2024-09-21T22:50:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0530000000000003e-06 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294725+0000", 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807999999999997e-05 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295530+0000", 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080478500000000003 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330672+0000", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035142709000000001 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330748+0000", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5794e-05 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330794+0000", 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5654e-05 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35857 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5247844989999999, 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035441063000000002, 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35857 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295636+0000", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295642+0000", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295662+0000", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9775e-05 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295677+0000", 2024-09-21T22:50:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4606e-05 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330937+0000", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035260540999999999 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330998+0000", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0763000000000001e-05 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331076+0000", 2024-09-21T22:50:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8197999999999999e-05 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35858 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5247239920000002, 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035527577999999997, 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35858 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295697+0000", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295703+0000", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.993e-06 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295740+0000", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6984999999999999e-05 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.296513+0000", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077355700000000002 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331106+0000", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034593169999999999 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331179+0000", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2705000000000003e-05 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331223+0000", 2024-09-21T22:50:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4091000000000002e-05 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:49450 3.1d 3:bc968670:::rbd_data.111e5668b648.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.99349593300000005, 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041268020000000002, 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 49450 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826926+0000", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834639+0000", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077131559999999997 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834665+0000", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6054000000000002e-05 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834681+0000", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6161e-05 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834780+0000", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8319999999999994e-05 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834942+0000", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016233 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.836568+0000", 2024-09-21T22:50:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00162598 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.867947+0000", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031378915 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868057+0000", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109911 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868128+0000", 2024-09-21T22:50:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1236999999999996e-05 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868192+0000", 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3806000000000005e-05 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:50.907 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3085 read 0xba7208 thru 0xbb2506 (0xb2ff bytes) 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3087 punch from 0x1c81c78 to 0x1c85bb9, (0x3f41 bytes) 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3088 punch from 0x2404ae6 to 0x24087e2, (0x3cfc bytes) 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3089 write 0x62e2f3c thru 0x62e2fba (0x7f bytes) 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3090 write 0x77c9d77 thru 0x77d24a1 (0x872b bytes) 2024-09-21T22:50:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:3091 punch from 0xfef174 to 0xff9816, (0xa6a2 bytes) 2024-09-21T22:50:50.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:50.910 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:50.904+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:50.932 INFO:teuthology.orchestra.run.smithi094.stdout:3092 writesame 0xe3e8969 thru 0xe3ed5c9 data_size 0x1(0x4c61 bytes) 2024-09-21T22:50:50.932 INFO:teuthology.orchestra.run.smithi094.stdout:3094 writesame 0x67552fa thru 0x6758ee0 data_size 0x1(0x3be7 bytes) 2024-09-21T22:50:50.945 INFO:teuthology.orchestra.run.smithi094.stdout:3095 punch from 0x2b7ae38 to 0x2b807d4, (0x599c bytes) 2024-09-21T22:50:50.945 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.946 INFO:teuthology.orchestra.run.smithi094.stdout:3098 writesame 0x5364464 thru 0x5370c18 data_size 0x19(0xc7b5 bytes) 2024-09-21T22:50:50.950 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:50.950 INFO:teuthology.orchestra.run.smithi094.stdout:3101 write 0xbfa6a41 thru 0xbfb2684 (0xbc44 bytes) 2024-09-21T22:50:50.958 INFO:teuthology.orchestra.run.smithi094.stdout:3102 write 0x46e0ef3 thru 0x46eaa20 (0x9b2e bytes) 2024-09-21T22:50:50.964 INFO:teuthology.orchestra.run.smithi094.stdout:3103 read 0xb4ca3f9 thru 0xb4ca4ae (0xb6 bytes) 2024-09-21T22:50:50.964 INFO:teuthology.orchestra.run.smithi094.stdout:3104 clone 16 order 19 su 4096 sc 13 2024-09-21T22:50:51.009 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:51.008+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:51.033 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:51.033 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:51.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:51.103 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:50:51.110 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:51.104+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:51.211 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:51.208+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:leaving image image_client.0-clone15 intact 2024-09-21T22:50:51.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:50:51.217 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:51.217 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T22:50:51.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T22:50:51.219 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:51.219 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:51.219 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:51.219 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:51.219 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:51.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:51.321 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:51.320+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.465682504, 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9206750289999999, 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:50:51.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:51.416 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.920610645, 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:50:51.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9205471950000002, 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:51.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:51.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9204840169999997, 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9204276770000002, 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:50:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:50:51.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:50:51.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:50:51.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9203859349999997, 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:50:51.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:50:51.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9203253340000002, 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:50:51.438 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:50:51.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:50:51.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.920244898, 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.443 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:50:51.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9201687879999998, 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.447 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:50:51.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:50:51.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9200817150000002, 2024-09-21T22:50:51.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:51.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.920038227, 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:50:51.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9200045890000004, 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9199708229999999, 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9199395389999996, 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9198941840000003, 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9198626880000003, 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:50:51.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:50:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9198279070000002, 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:51.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:50:51.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:50:51.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9018823759999997, 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:51.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9018213319999999, 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:51.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:51.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:51.495 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:51.495 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T22:50:51.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:51.496 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:51.488+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:51.542 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 56, 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "56013142-0739-4aee-ba29-30eddcc2dbe0", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-21T22:49:36.309693+0000", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-21T22:50:51.095131+0000", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-21T22:50:33.163047+0000", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-21T22:49:39.319546+0000", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:50:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 6, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "jewel", 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-21T22:50:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:51.363777+0000", 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:50:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "19", 2024-09-21T22:50:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:50:51.547 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:50:51.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-21T22:50:51.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-21T22:50:51.550 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:55.226638+0000", 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:50:51.551 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.552 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "23", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 23, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:50:51.553 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:50:51.554 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:50:51.555 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:50:05.084190+0000", 2024-09-21T22:50:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-21T22:50:51.557 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "56", 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:50:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 18, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 56, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:50:51.559 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:50:51.560 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.5, 2024-09-21T22:50:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.5, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.5, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.5, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:50:51.562 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ecfcea04-7198-4c57-bc74-45aaf791e3b7", 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 44, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.563 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/2211646084", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/2211646084", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/2211646084", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/2211646084", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.567 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b6b749e6-4a66-4d8b-bf50-c9634a6927d5", 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 42, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.568 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.572 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1102146551", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1102146551", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1102146551", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1102146551", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "4e90da65-2fdd-4c85-a207-70d5f22c95a4", 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.573 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 20, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.577 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/2482161893", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/2482161893", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/2482161893", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/2482161893", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "3c97651a-402c-4171-9648-c615dd754c94", 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 45, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.579 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.580 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2344760438 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.583 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/2344760438", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/2344760438", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/2344760438", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/2344760438", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b935b00e-0d91-4f2b-9f2d-2c2be1144ca8", 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.584 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 24, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6819", 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.585 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6821", 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6823", 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.586 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6826", 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6832", 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.587 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6833", 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6828", 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6830", 2024-09-21T22:50:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6821/1105568802", 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6826/1105568802", 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6833/1105568802", 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6830/1105568802", 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:50:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "559dea61-5abd-423e-9dfb-cf92f9e6d4a4", 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 16, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 24, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6818", 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6820", 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6822", 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6824", 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.592 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6829", 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6831", 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.593 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6825", 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6827", 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6820/2691947692", 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6824/2691947692", 2024-09-21T22:50:51.594 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2691947692", 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6827/2691947692", 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "aeee922d-53c2-41f4-9433-8a9691f3e674", 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.595 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 15, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 23, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 42, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 45, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 24, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6810", 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.596 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6811", 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6812", 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6813", 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6816", 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6817", 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6814", 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6815", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6811/3470582226", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6813/3470582226", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6817/3470582226", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6815/3470582226", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.600 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b40a3234-6950-4a73-9a5c-f0c086450816", 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 24, 2024-09-21T22:50:51.601 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6800", 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6801", 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.602 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6802", 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6803", 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.603 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6806", 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6807", 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6804", 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6805", 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.605 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6801/1713641021", 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6803/1713641021", 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1713641021", 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1713641021", 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:50:51.606 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.846676+0000", 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.607 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:47.117051+0000", 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.608 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.836357+0000", 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.351920+0000", 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:50:51.609 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.975489+0000", 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.610 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.838876+0000", 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.842751+0000", 2024-09-21T22:50:51.611 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 24 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.784805+0000", 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-21T22:50:51.612 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [], 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [], 2024-09-21T22:50:51.613 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [], 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:51.614 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:51.616 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': '56013142-0739-4aee-ba29-30eddcc2dbe0', 'created': '2024-09-21T22:49:36.309693+0000', 'modified': '2024-09-21T22:50:51.095131+0000', 'last_up_change': '2024-09-21T22:50:33.163047+0000', 'last_in_change': '2024-09-21T22:49:39.319546+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T22:49:51.363777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T22:49:55.226638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T22:50:05.084190+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ecfcea04-7198-4c57-bc74-45aaf791e3b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 2211646084}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 2211646084}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 2211646084}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 2211646084}]}, 'public_addr': '172.21.15.94:6809/2211646084', 'cluster_addr': '172.21.15.94:6811/2211646084', 'heartbeat_back_addr': '172.21.15.94:6815/2211646084', 'heartbeat_front_addr': '172.21.15.94:6813/2211646084', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b6b749e6-4a66-4d8b-bf50-c9634a6927d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1102146551}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1102146551}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1102146551}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1102146551}]}, 'public_addr': '172.21.15.94:6825/1102146551', 'cluster_addr': '172.21.15.94:6827/1102146551', 'heartbeat_back_addr': '172.21.15.94:6831/1102146551', 'heartbeat_front_addr': '172.21.15.94:6829/1102146551', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4e90da65-2fdd-4c85-a207-70d5f22c95a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 2482161893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 2482161893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 2482161893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 2482161893}]}, 'public_addr': '172.21.15.94:6817/2482161893', 'cluster_addr': '172.21.15.94:6819/2482161893', 'heartbeat_back_addr': '172.21.15.94:6823/2482161893', 'heartbeat_front_addr': '172.21.15.94:6821/2482161893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3c97651a-402c-4171-9648-c615dd754c94', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 2344760438}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 2344760438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 2344760438}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 2344760438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 2344760438}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 2344760438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 2344760438}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 2344760438}]}, 'public_addr': '172.21.15.94:6801/2344760438', 'cluster_addr': '172.21.15.94:6803/2344760438', 'heartbeat_back_addr': '172.21.15.94:6807/2344760438', 'heartbeat_front_addr': '172.21.15.94:6805/2344760438', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b935b00e-0d91-4f2b-9f2d-2c2be1144ca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6819', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 1105568802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6823', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6826', 'nonce': 1105568802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 1105568802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6830', 'nonce': 1105568802}]}, 'public_addr': '172.21.15.99:6821/1105568802', 'cluster_addr': '172.21.15.99:6826/1105568802', 'heartbeat_back_addr': '172.21.15.99:6833/1105568802', 'heartbeat_front_addr': '172.21.15.99:6830/1105568802', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '559dea61-5abd-423e-9dfb-cf92f9e6d4a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6820', 'nonce': 2691947692}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6824', 'nonce': 2691947692}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6829', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2691947692}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6825', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2691947692}]}, 'public_addr': '172.21.15.99:6820/2691947692', 'cluster_addr': '172.21.15.99:6824/2691947692', 'heartbeat_back_addr': '172.21.15.99:6831/2691947692', 'heartbeat_front_addr': '172.21.15.99:6827/2691947692', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'aeee922d-53c2-41f4-9433-8a9691f3e674', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 23, 'up_from': 42, 'up_thru': 45, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3470582226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 3470582226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3470582226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3470582226}]}, 'public_addr': '172.21.15.99:6811/3470582226', 'cluster_addr': '172.21.15.99:6813/3470582226', 'heartbeat_back_addr': '172.21.15.99:6817/3470582226', 'heartbeat_front_addr': '172.21.15.99:6815/3470582226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b40a3234-6950-4a73-9a5c-f0c086450816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1713641021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1713641021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1713641021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1713641021}]}, 'public_addr': '172.21.15.99:6801/1713641021', 'cluster_addr': '172.21.15.99:6803/1713641021', 'heartbeat_back_addr': '172.21.15.99:6807/1713641021', 'heartbeat_front_addr': '172.21.15.99:6805/1713641021', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.846676+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:47.117051+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.836357+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.351920+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.975489+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.838876+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.842751+0000', 'dead_epoch': 24}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.784805+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T22:50:51.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:50:51.617 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:51.612+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:51.667 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:51.667 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:51.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:50:51.716 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:51.712+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:51.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:51.817 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:51.816+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:51.845 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:51.845 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:51.845 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:51.846 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:51.846 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:51.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:50:51.866 INFO:teuthology.orchestra.run.smithi094.stdout:3105 read 0x3b2d5bb thru 0x3b3b14f (0xdb95 bytes) 2024-09-21T22:50:51.867 INFO:teuthology.orchestra.run.smithi094.stdout:3106 read 0x67d0d95 thru 0x67d5b63 (0x4dcf bytes) 2024-09-21T22:50:51.869 INFO:teuthology.orchestra.run.smithi094.stdout:3108 read 0x684f143 thru 0x68531fb (0x40b9 bytes) 2024-09-21T22:50:51.888 INFO:teuthology.orchestra.run.smithi094.stdout:3110 trunc from 0xe3ed5ca to 0x172f0c1 2024-09-21T22:50:51.918 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:51.912+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:51.978 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:51.978 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:52.000 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:52.019 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:52.016+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:52.019 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:52.023 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:50:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:50:52.037 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:52.041 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-21T22:50:51.285048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238928779,"num_objects":1422,"num_object_clones":457,"num_object_copies":2844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1422,"num_whiteouts":704,"num_read":67670,"num_read_kb":148330,"num_write":11853,"num_write_kb":100831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":413163,"num_keys_recovered":33,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9603,"ondisk_log_size":9603,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409048,"kb_used_data":119544,"kb_used_omap":15,"kb_used_meta":289264,"kb_avail":754565672,"statfs":{"total":773094113280,"available":772675248128,"internally_reserved":0,"allocated":122413056,"data_stored":146169411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15556,"internal_metadata":296207164},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37260487,"num_objects":319,"num_object_clones":137,"num_object_copies":638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":159,"num_read":17638,"num_read_kb":32713,"num_write":1886,"num_write_kb":21440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.209405"},"pg_stats":[{"pgid":"3.1f","version":"55'458","reported_seq":1318,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.431197+0000","last_change":"2024-09-21T22:50:09.054251+0000","last_active":"2024-09-21T22:50:50.431197+0000","last_peered":"2024-09-21T22:50:50.431197+0000","last_clean":"2024-09-21T22:50:50.431197+0000","last_became_active":"2024-09-21T22:50:07.041291+0000","last_became_peered":"2024-09-21T22:50:07.041291+0000","last_unstale":"2024-09-21T22:50:50.431197+0000","last_undegraded":"2024-09-21T22:50:50.431197+0000","last_fullsized":"2024-09-21T22:50:50.431197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:43.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730179999999999,"stat_sum":{"num_bytes":11435584,"num_objects":58,"num_object_clones":20,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":2044,"num_read_kb":6169,"num_write":604,"num_write_kb":4610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"53'248","reported_seq":1471,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111650+0000","last_change":"2024-09-21T22:50:37.035523+0000","last_active":"2024-09-21T22:50:48.111650+0000","last_peered":"2024-09-21T22:50:48.111650+0000","last_clean":"2024-09-21T22:50:48.111650+0000","last_became_active":"2024-09-21T22:50:37.030715+0000","last_became_peered":"2024-09-21T22:50:37.030715+0000","last_unstale":"2024-09-21T22:50:48.111650+0000","last_undegraded":"2024-09-21T22:50:48.111650+0000","last_fullsized":"2024-09-21T22:50:48.111650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:08.143803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020037760000000001,"stat_sum":{"num_bytes":9208276,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":23,"num_read":1836,"num_read_kb":3412,"num_write":222,"num_write_kb":1260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":85270,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"53'634","reported_seq":2164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.340096+0000","last_change":"2024-09-21T22:50:09.045034+0000","last_active":"2024-09-21T22:50:46.340096+0000","last_peered":"2024-09-21T22:50:46.340096+0000","last_clean":"2024-09-21T22:50:46.340096+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:50:46.340096+0000","last_undegraded":"2024-09-21T22:50:46.340096+0000","last_fullsized":"2024-09-21T22:50:46.340096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:58:10.296348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010806819999999999,"stat_sum":{"num_bytes":13241702,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3863,"num_read_kb":13029,"num_write":961,"num_write_kb":6830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"53'200","reported_seq":664,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.422122+0000","last_change":"2024-09-21T22:50:09.046692+0000","last_active":"2024-09-21T22:50:46.422122+0000","last_peered":"2024-09-21T22:50:46.422122+0000","last_clean":"2024-09-21T22:50:46.422122+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:50:46.422122+0000","last_undegraded":"2024-09-21T22:50:46.422122+0000","last_fullsized":"2024-09-21T22:50:46.422122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:01.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002495611,"stat_sum":{"num_bytes":1890340,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":844,"num_read_kb":1684,"num_write":205,"num_write_kb":1580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'773","reported_seq":4504,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.571156+0000","last_change":"2024-09-21T22:50:09.054590+0000","last_active":"2024-09-21T22:50:49.571156+0000","last_peered":"2024-09-21T22:50:49.571156+0000","last_clean":"2024-09-21T22:50:49.571156+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:50:49.571156+0000","last_undegraded":"2024-09-21T22:50:49.571156+0000","last_fullsized":"2024-09-21T22:50:49.571156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:01:04.393413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001622592,"stat_sum":{"num_bytes":5660241,"num_objects":50,"num_object_clones":19,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":7151,"num_read_kb":9678,"num_write":1308,"num_write_kb":3846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"55'588","reported_seq":2789,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.433930+0000","last_change":"2024-09-21T22:50:09.031950+0000","last_active":"2024-09-21T22:50:50.433930+0000","last_peered":"2024-09-21T22:50:50.433930+0000","last_clean":"2024-09-21T22:50:50.433930+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:50:50.433930+0000","last_undegraded":"2024-09-21T22:50:50.433930+0000","last_fullsized":"2024-09-21T22:50:50.433930+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:43:45.754522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002297323,"stat_sum":{"num_bytes":7681598,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":4327,"num_read_kb":5871,"num_write":917,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"53'180","reported_seq":3174,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.475125+0000","last_change":"2024-09-21T22:50:09.032544+0000","last_active":"2024-09-21T22:50:49.475125+0000","last_peered":"2024-09-21T22:50:49.475125+0000","last_clean":"2024-09-21T22:50:49.475125+0000","last_became_active":"2024-09-21T22:50:07.033757+0000","last_became_peered":"2024-09-21T22:50:07.033757+0000","last_unstale":"2024-09-21T22:50:49.475125+0000","last_undegraded":"2024-09-21T22:50:49.475125+0000","last_fullsized":"2024-09-21T22:50:49.475125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:35.473083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084097000000000002,"stat_sum":{"num_bytes":9760901,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":2954,"num_read_kb":9020,"num_write":161,"num_write_kb":1773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"53'199","reported_seq":2282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111347+0000","last_change":"2024-09-21T22:50:09.027841+0000","last_active":"2024-09-21T22:50:48.111347+0000","last_peered":"2024-09-21T22:50:48.111347+0000","last_clean":"2024-09-21T22:50:48.111347+0000","last_became_active":"2024-09-21T22:50:07.034008+0000","last_became_peered":"2024-09-21T22:50:07.034008+0000","last_unstale":"2024-09-21T22:50:48.111347+0000","last_undegraded":"2024-09-21T22:50:48.111347+0000","last_fullsized":"2024-09-21T22:50:48.111347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:01.434438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020218739999999999,"stat_sum":{"num_bytes":8096084,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":2511,"num_read_kb":2611,"num_write":276,"num_write_kb":3621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"53'183","reported_seq":754,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.183488+0000","last_change":"2024-09-21T22:50:09.054285+0000","last_active":"2024-09-21T22:50:49.183488+0000","last_peered":"2024-09-21T22:50:49.183488+0000","last_clean":"2024-09-21T22:50:49.183488+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:50:49.183488+0000","last_undegraded":"2024-09-21T22:50:49.183488+0000","last_fullsized":"2024-09-21T22:50:49.183488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:11:35.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013918940000000001,"stat_sum":{"num_bytes":3491990,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":601,"num_read_kb":1605,"num_write":184,"num_write_kb":1528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"53'172","reported_seq":1236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.232437+0000","last_change":"2024-09-21T22:50:09.045475+0000","last_active":"2024-09-21T22:50:46.232437+0000","last_peered":"2024-09-21T22:50:46.232437+0000","last_clean":"2024-09-21T22:50:46.232437+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:50:46.232437+0000","last_undegraded":"2024-09-21T22:50:46.232437+0000","last_fullsized":"2024-09-21T22:50:46.232437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:36:39.340446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504238,"stat_sum":{"num_bytes":4208122,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":1026,"num_read_kb":2043,"num_write":150,"num_write_kb":2070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'184","reported_seq":796,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.512606+0000","last_change":"2024-09-21T22:50:09.062094+0000","last_active":"2024-09-21T22:50:49.512606+0000","last_peered":"2024-09-21T22:50:49.512606+0000","last_clean":"2024-09-21T22:50:49.512606+0000","last_became_active":"2024-09-21T22:50:07.039164+0000","last_became_peered":"2024-09-21T22:50:07.039164+0000","last_unstale":"2024-09-21T22:50:49.512606+0000","last_undegraded":"2024-09-21T22:50:49.512606+0000","last_fullsized":"2024-09-21T22:50:49.512606+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:40.958004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081156800000000003,"stat_sum":{"num_bytes":5119152,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":571,"num_read_kb":5629,"num_write":176,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"53'174","reported_seq":1083,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.479711+0000","last_change":"2024-09-21T22:50:09.032016+0000","last_active":"2024-09-21T22:50:49.479711+0000","last_peered":"2024-09-21T22:50:49.479711+0000","last_clean":"2024-09-21T22:50:49.479711+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:50:49.479711+0000","last_undegraded":"2024-09-21T22:50:49.479711+0000","last_fullsized":"2024-09-21T22:50:49.479711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:30:53.485963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021866500000000001,"stat_sum":{"num_bytes":15616198,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":21,"num_read":872,"num_read_kb":1984,"num_write":173,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.193131+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:50:46.193131+0000","last_peered":"2024-09-21T22:50:46.193131+0000","last_clean":"2024-09-21T22:50:46.193131+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:50:46.193131+0000","last_undegraded":"2024-09-21T22:50:46.193131+0000","last_fullsized":"2024-09-21T22:50:46.193131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'161","reported_seq":1080,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.363518+0000","last_change":"2024-09-21T22:50:09.053734+0000","last_active":"2024-09-21T22:50:50.363518+0000","last_peered":"2024-09-21T22:50:50.363518+0000","last_clean":"2024-09-21T22:50:50.363518+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:50:50.363518+0000","last_undegraded":"2024-09-21T22:50:50.363518+0000","last_fullsized":"2024-09-21T22:50:50.363518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:22.397507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003049,"stat_sum":{"num_bytes":4887032,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":881,"num_read_kb":2509,"num_write":157,"num_write_kb":1836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.088209+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:50:49.088209+0000","last_peered":"2024-09-21T22:50:49.088209+0000","last_clean":"2024-09-21T22:50:49.088209+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:50:49.088209+0000","last_undegraded":"2024-09-21T22:50:49.088209+0000","last_fullsized":"2024-09-21T22:50:49.088209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'328","reported_seq":1192,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111623+0000","last_change":"2024-09-21T22:50:09.027492+0000","last_active":"2024-09-21T22:50:48.111623+0000","last_peered":"2024-09-21T22:50:48.111623+0000","last_clean":"2024-09-21T22:50:48.111623+0000","last_became_active":"2024-09-21T22:50:07.028763+0000","last_became_peered":"2024-09-21T22:50:07.028763+0000","last_unstale":"2024-09-21T22:50:48.111623+0000","last_undegraded":"2024-09-21T22:50:48.111623+0000","last_fullsized":"2024-09-21T22:50:48.111623+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:13:49.436599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018249729999999999,"stat_sum":{"num_bytes":3094124,"num_objects":49,"num_object_clones":17,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":1963,"num_read_kb":3291,"num_write":386,"num_write_kb":1614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.106514+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:50:49.106514+0000","last_peered":"2024-09-21T22:50:49.106514+0000","last_clean":"2024-09-21T22:50:49.106514+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:50:49.106514+0000","last_undegraded":"2024-09-21T22:50:49.106514+0000","last_fullsized":"2024-09-21T22:50:49.106514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'335","reported_seq":1980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.234623+0000","last_change":"2024-09-21T22:50:09.032823+0000","last_active":"2024-09-21T22:50:50.234623+0000","last_peered":"2024-09-21T22:50:50.234623+0000","last_clean":"2024-09-21T22:50:50.234623+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:50:50.234623+0000","last_undegraded":"2024-09-21T22:50:50.234623+0000","last_fullsized":"2024-09-21T22:50:50.234623+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:23.125057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093496699999999998,"stat_sum":{"num_bytes":6271008,"num_objects":49,"num_object_clones":17,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":21,"num_read":2386,"num_read_kb":2343,"num_write":434,"num_write_kb":1873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.088325+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:50:49.088325+0000","last_peered":"2024-09-21T22:50:49.088325+0000","last_clean":"2024-09-21T22:50:49.088325+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:50:49.088325+0000","last_undegraded":"2024-09-21T22:50:49.088325+0000","last_fullsized":"2024-09-21T22:50:49.088325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'162","reported_seq":309,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.256493+0000","last_change":"2024-09-21T22:50:09.046691+0000","last_active":"2024-09-21T22:50:46.256493+0000","last_peered":"2024-09-21T22:50:46.256493+0000","last_clean":"2024-09-21T22:50:46.256493+0000","last_became_active":"2024-09-21T22:50:07.039630+0000","last_became_peered":"2024-09-21T22:50:07.039630+0000","last_unstale":"2024-09-21T22:50:46.256493+0000","last_undegraded":"2024-09-21T22:50:46.256493+0000","last_fullsized":"2024-09-21T22:50:46.256493+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:35:33.507805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021691330000000002,"stat_sum":{"num_bytes":4434034,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":115,"num_read_kb":4728,"num_write":155,"num_write_kb":5000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'247","reported_seq":1015,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.386152+0000","last_change":"2024-09-21T22:50:09.030568+0000","last_active":"2024-09-21T22:50:50.386152+0000","last_peered":"2024-09-21T22:50:50.386152+0000","last_clean":"2024-09-21T22:50:50.386152+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:50:50.386152+0000","last_undegraded":"2024-09-21T22:50:50.386152+0000","last_fullsized":"2024-09-21T22:50:50.386152+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:05:22.432489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079408,"stat_sum":{"num_bytes":12776455,"num_objects":41,"num_object_clones":16,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":1847,"num_read_kb":4429,"num_write":238,"num_write_kb":2873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.090220+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:50:46.090220+0000","last_peered":"2024-09-21T22:50:46.090220+0000","last_clean":"2024-09-21T22:50:46.090220+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:50:46.090220+0000","last_undegraded":"2024-09-21T22:50:46.090220+0000","last_fullsized":"2024-09-21T22:50:46.090220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"55'423","reported_seq":1934,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.395819+0000","last_change":"2024-09-21T22:50:09.032584+0000","last_active":"2024-09-21T22:50:50.395819+0000","last_peered":"2024-09-21T22:50:50.395819+0000","last_clean":"2024-09-21T22:50:50.395819+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:50:50.395819+0000","last_undegraded":"2024-09-21T22:50:50.395819+0000","last_fullsized":"2024-09-21T22:50:50.395819+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:23.772367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076847199999999997,"stat_sum":{"num_bytes":5079764,"num_objects":58,"num_object_clones":21,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":3131,"num_read_kb":5536,"num_write":676,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.718267+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:50:49.718267+0000","last_peered":"2024-09-21T22:50:49.718267+0000","last_clean":"2024-09-21T22:50:49.718267+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:50:49.718267+0000","last_undegraded":"2024-09-21T22:50:49.718267+0000","last_fullsized":"2024-09-21T22:50:49.718267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"53'212","reported_seq":993,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.490298+0000","last_change":"2024-09-21T22:50:09.053334+0000","last_active":"2024-09-21T22:50:49.490298+0000","last_peered":"2024-09-21T22:50:49.490298+0000","last_clean":"2024-09-21T22:50:49.490298+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:50:49.490298+0000","last_undegraded":"2024-09-21T22:50:49.490298+0000","last_fullsized":"2024-09-21T22:50:49.490298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:45.872289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068683399999999999,"stat_sum":{"num_bytes":12276009,"num_objects":55,"num_object_clones":19,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":798,"num_read_kb":2721,"num_write":179,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.105901+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:50:49.105901+0000","last_peered":"2024-09-21T22:50:49.105901+0000","last_clean":"2024-09-21T22:50:49.105901+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:50:49.105901+0000","last_undegraded":"2024-09-21T22:50:49.105901+0000","last_fullsized":"2024-09-21T22:50:49.105901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'430","reported_seq":1853,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111439+0000","last_change":"2024-09-21T22:50:09.026491+0000","last_active":"2024-09-21T22:50:48.111439+0000","last_peered":"2024-09-21T22:50:48.111439+0000","last_clean":"2024-09-21T22:50:48.111439+0000","last_became_active":"2024-09-21T22:50:07.028774+0000","last_became_peered":"2024-09-21T22:50:07.028774+0000","last_unstale":"2024-09-21T22:50:48.111439+0000","last_undegraded":"2024-09-21T22:50:48.111439+0000","last_fullsized":"2024-09-21T22:50:48.111439+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:43:54.966406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010115009999999999,"stat_sum":{"num_bytes":11584034,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":2549,"num_read_kb":6376,"num_write":595,"num_write_kb":4383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.090170+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:50:46.090170+0000","last_peered":"2024-09-21T22:50:46.090170+0000","last_clean":"2024-09-21T22:50:46.090170+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:50:46.090170+0000","last_undegraded":"2024-09-21T22:50:46.090170+0000","last_fullsized":"2024-09-21T22:50:46.090170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.105961+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:50:49.105961+0000","last_peered":"2024-09-21T22:50:49.105961+0000","last_clean":"2024-09-21T22:50:49.105961+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:50:49.105961+0000","last_undegraded":"2024-09-21T22:50:49.105961+0000","last_fullsized":"2024-09-21T22:50:49.105961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"53'195","reported_seq":1754,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111599+0000","last_change":"2024-09-21T22:50:09.026886+0000","last_active":"2024-09-21T22:50:48.111599+0000","last_peered":"2024-09-21T22:50:48.111599+0000","last_clean":"2024-09-21T22:50:48.111599+0000","last_became_active":"2024-09-21T22:50:07.032070+0000","last_became_peered":"2024-09-21T22:50:07.032070+0000","last_unstale":"2024-09-21T22:50:48.111599+0000","last_undegraded":"2024-09-21T22:50:48.111599+0000","last_fullsized":"2024-09-21T22:50:48.111599+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:28:08.107191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001477641,"stat_sum":{"num_bytes":11244014,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":1958,"num_read_kb":1608,"num_write":183,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"53'162","reported_seq":930,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.095387+0000","last_change":"2024-09-21T22:50:09.030512+0000","last_active":"2024-09-21T22:50:49.095387+0000","last_peered":"2024-09-21T22:50:49.095387+0000","last_clean":"2024-09-21T22:50:49.095387+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:50:49.095387+0000","last_undegraded":"2024-09-21T22:50:49.095387+0000","last_fullsized":"2024-09-21T22:50:49.095387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:21:16.743737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00126947,"stat_sum":{"num_bytes":3844790,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":1038,"num_read_kb":3263,"num_write":134,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'121","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:47.184869+0000","last_change":"2024-09-21T22:50:34.240149+0000","last_active":"2024-09-21T22:50:47.184869+0000","last_peered":"2024-09-21T22:50:47.184869+0000","last_clean":"2024-09-21T22:50:47.184869+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:50:47.184869+0000","last_undegraded":"2024-09-21T22:50:47.184869+0000","last_fullsized":"2024-09-21T22:50:47.184869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:26.850266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092677,"stat_sum":{"num_bytes":2427320,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":95,"num_read_kb":66,"num_write":109,"num_write_kb":436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"53'186","reported_seq":1020,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-21T22:50:48.111435+0000","last_change":"2024-09-21T22:50:09.027738+0000","last_active":"2024-09-21T22:50:48.111435+0000","last_peered":"2024-09-21T22:50:48.111435+0000","last_clean":"2024-09-21T22:50:48.111435+0000","last_became_active":"2024-09-21T22:50:07.037670+0000","last_became_peered":"2024-09-21T22:50:07.037670+0000","last_unstale":"2024-09-21T22:50:48.111435+0000","last_undegraded":"2024-09-21T22:50:48.111435+0000","last_fullsized":"2024-09-21T22:50:48.111435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:54.817901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002153528,"stat_sum":{"num_bytes":4036523,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":26,"num_read":812,"num_read_kb":2662,"num_write":160,"num_write_kb":2196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"53'147","reported_seq":425,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:46.161222+0000","last_change":"2024-09-21T22:50:09.044903+0000","last_active":"2024-09-21T22:50:46.161222+0000","last_peered":"2024-09-21T22:50:46.161222+0000","last_clean":"2024-09-21T22:50:46.161222+0000","last_became_active":"2024-09-21T22:50:07.035007+0000","last_became_peered":"2024-09-21T22:50:07.035007+0000","last_unstale":"2024-09-21T22:50:46.161222+0000","last_undegraded":"2024-09-21T22:50:46.161222+0000","last_fullsized":"2024-09-21T22:50:46.161222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:30.254029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261420000000001,"stat_sum":{"num_bytes":4464139,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":235,"num_read_kb":3276,"num_write":141,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"55'242","reported_seq":1949,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.461350+0000","last_change":"2024-09-21T22:50:09.064319+0000","last_active":"2024-09-21T22:50:50.461350+0000","last_peered":"2024-09-21T22:50:50.461350+0000","last_clean":"2024-09-21T22:50:50.461350+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:50:50.461350+0000","last_undegraded":"2024-09-21T22:50:50.461350+0000","last_fullsized":"2024-09-21T22:50:50.461350+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:38.901827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024500839999999999,"stat_sum":{"num_bytes":4824514,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2687,"num_read_kb":4930,"num_write":241,"num_write_kb":2348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"55'259","reported_seq":1914,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.053388+0000","last_change":"2024-09-21T22:50:09.064534+0000","last_active":"2024-09-21T22:50:50.053388+0000","last_peered":"2024-09-21T22:50:50.053388+0000","last_clean":"2024-09-21T22:50:50.053388+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:50:50.053388+0000","last_undegraded":"2024-09-21T22:50:50.053388+0000","last_fullsized":"2024-09-21T22:50:50.053388+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:35:56.299911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002506234,"stat_sum":{"num_bytes":14822804,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":2399,"num_read_kb":6358,"num_write":320,"num_write_kb":3778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"55'245","reported_seq":4679,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.232779+0000","last_change":"2024-09-21T22:50:09.064820+0000","last_active":"2024-09-21T22:50:50.232779+0000","last_peered":"2024-09-21T22:50:50.232779+0000","last_clean":"2024-09-21T22:50:50.232779+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:50:50.232779+0000","last_undegraded":"2024-09-21T22:50:50.232779+0000","last_fullsized":"2024-09-21T22:50:50.232779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:06.813259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021047100000000001,"stat_sum":{"num_bytes":10013055,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":5328,"num_read_kb":7873,"num_write":232,"num_write_kb":1398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"53'316","reported_seq":1701,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T22:50:47.188200+0000","last_change":"2024-09-21T22:50:37.035011+0000","last_active":"2024-09-21T22:50:47.188200+0000","last_peered":"2024-09-21T22:50:47.188200+0000","last_clean":"2024-09-21T22:50:47.188200+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:50:47.188200+0000","last_undegraded":"2024-09-21T22:50:47.188200+0000","last_fullsized":"2024-09-21T22:50:47.188200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:56.655349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001972101,"stat_sum":{"num_bytes":7973912,"num_objects":47,"num_object_clones":17,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":2823,"num_read_kb":3814,"num_write":298,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"53'687","reported_seq":1082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:49.808239+0000","last_change":"2024-09-21T22:50:09.062805+0000","last_active":"2024-09-21T22:50:49.808239+0000","last_peered":"2024-09-21T22:50:49.808239+0000","last_clean":"2024-09-21T22:50:49.808239+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:50:49.808239+0000","last_undegraded":"2024-09-21T22:50:49.808239+0000","last_fullsized":"2024-09-21T22:50:49.808239+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:52:05.118846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014275189999999999,"stat_sum":{"num_bytes":6103001,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2874,"num_read_kb":5760,"num_write":1014,"num_write_kb":4677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"55'292","reported_seq":2526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.395347+0000","last_change":"2024-09-21T22:50:09.053339+0000","last_active":"2024-09-21T22:50:50.395347+0000","last_peered":"2024-09-21T22:50:50.395347+0000","last_clean":"2024-09-21T22:50:50.395347+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:50:50.395347+0000","last_undegraded":"2024-09-21T22:50:50.395347+0000","last_fullsized":"2024-09-21T22:50:50.395347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:58:45.112867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066547199999999996,"stat_sum":{"num_bytes":5494161,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":3542,"num_read_kb":7971,"num_write":278,"num_write_kb":2697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'340","reported_seq":1363,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-21T22:50:50.395516+0000","last_change":"2024-09-21T22:50:09.062275+0000","last_active":"2024-09-21T22:50:50.395516+0000","last_peered":"2024-09-21T22:50:50.395516+0000","last_clean":"2024-09-21T22:50:50.395516+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:50:50.395516+0000","last_undegraded":"2024-09-21T22:50:50.395516+0000","last_fullsized":"2024-09-21T22:50:50.395516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:44.105359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092760900000000003,"stat_sum":{"num_bytes":7277511,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":29,"num_read":1493,"num_read_kb":5981,"num_write":425,"num_write_kb":4031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":238338392,"num_objects":1418,"num_object_clones":457,"num_object_copies":2836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":704,"num_read":67554,"num_read_kb":148230,"num_write":11692,"num_write_kb":98413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":413163,"num_keys_recovered":33,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110211072,"data_stored":136899668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9483,"ondisk_log_size":9483,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476750,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52372,"kb_used_data":17024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319468,"statfs":{"total":96636764160,"available":96583135232,"internally_reserved":0,"allocated":17432576,"data_stored":24905726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56648,"kb_used_data":17144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315192,"statfs":{"total":96636764160,"available":96578756608,"internally_reserved":0,"allocated":17555456,"data_stored":19711545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626436,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29960,"kb_used_data":1840,"kb_used_omap":4,"kb_used_meta":28091,"kb_avail":94341880,"statfs":{"total":96636764160,"available":96606085120,"internally_reserved":0,"allocated":1884160,"data_stored":1714280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":28765906},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76364,"kb_used_data":31212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94295476,"statfs":{"total":96636764160,"available":96558567424,"internally_reserved":0,"allocated":31961088,"data_stored":38884898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39344,"kb_used_data":3996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332496,"statfs":{"total":96636764160,"available":96596475904,"internally_reserved":0,"allocated":4091904,"data_stored":5440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51724,"kb_used_data":12220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320116,"statfs":{"total":96636764160,"available":96583798784,"internally_reserved":0,"allocated":12513280,"data_stored":13570459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":19584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":20054016,"data_stored":22735749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56028,"kb_used_data":16524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315812,"statfs":{"total":96636764160,"available":96579391488,"internally_reserved":0,"allocated":16920576,"data_stored":19206626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18780160,"data_stored":21822145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6324224,"data_stored":8012677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3477504,"data_stored":5173516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18710528,"data_stored":21146879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28303360,"data_stored":35689731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16998400,"data_stored":24752825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1282048,"data_stored":1453441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16334848,"data_stored":18848454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:50:52.044 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [3, 5] 2024-09-21T22:50:52.044 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1f', '3', '5'] 2024-09-21T22:50:52.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1f 3 5 2024-09-21T22:50:52.058 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:52.058 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:52.063 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 16.042430770999999, 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:50:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:50:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 16.038193201999999, 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:50:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 16.037998303999998, 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:50:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:50:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:50:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34552 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.8945168189999997, 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013168394, 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34552 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167254+0000", 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2320000000000003e-06 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167273+0000", 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9502999999999999e-05 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167323+0000", 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9527999999999997e-05 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167346+0000", 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3618999999999999e-05 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167513+0000", 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166992 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167758+0000", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000244622 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.169268+0000", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015101559999999999 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180086+0000", 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010817937999999999 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180137+0000", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0633999999999997e-05 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180295+0000", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015844899999999999 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180418+0000", 2024-09-21T22:50:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012272100000000001 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34557 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.8895740569999999, 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010897125000000001, 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34557 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172195+0000", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7099999999999999e-06 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172210+0000", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4823e-05 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172240+0000", 2024-09-21T22:50:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0361999999999998e-05 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172255+0000", 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552000000000001e-05 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172390+0000", 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013588399999999999 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172680+0000", 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028962999999999998 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.174751+0000", 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020706689999999998 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182868+0000", 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081170249999999999 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182936+0000", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8422000000000003e-05 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183021+0000", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5043000000000006e-05 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183089+0000", 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8004999999999997e-05 2024-09-21T22:50:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40788 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 190464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8887807419999998, 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012747681, 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40788 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172986+0000", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5499999999999998e-07 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172990+0000", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5820000000000002e-06 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172998+0000", 2024-09-21T22:50:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7679999999999997e-06 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173022+0000", 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3747000000000002e-05 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182235+0000", 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092129110000000007 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182364+0000", 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128589 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185733+0000", 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003369529 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40789 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.888733963, 2024-09-21T22:50:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010153383, 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40789 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173033+0000", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6400000000000003e-07 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173035+0000", 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8389999999999998e-06 2024-09-21T22:50:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173044+0000", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173049+0000", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5330000000000002e-06 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182955+0000", 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099058489999999996 2024-09-21T22:50:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183053+0000", 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8072000000000006e-05 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183186+0000", 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013232700000000001 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40790 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8887084869999997, 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010372809, 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40790 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7399999999999998e-07 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173061+0000", 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.853e-06 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173069+0000", 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3340000000000002e-06 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173077+0000", 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9780000000000005e-06 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183219+0000", 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010142004 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183319+0000", 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9908000000000006e-05 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183430+0000", 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111258 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40791 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8886783229999997, 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010578489, 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40791 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6899999999999998e-07 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173110+0000", 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1763000000000002e-05 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173120+0000", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8740000000000007e-06 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173135+0000", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5356999999999999e-05 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183465+0000", 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010329823 2024-09-21T22:50:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183558+0000", 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2762000000000004e-05 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183666+0000", 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108441 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40792 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8886167440000001, 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01076204, 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40792 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173150+0000", 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173163+0000", 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3438000000000001e-05 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173173+0000", 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5689999999999994e-06 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173180+0000", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7889999999999997e-06 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183703+0000", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010523341 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183800+0000", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6836999999999995e-05 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183911+0000", 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011145300000000001 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40793 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 223232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8885759760000003, 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010959662, 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40793 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173191+0000", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1399999999999997e-07 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173193+0000", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5830000000000001e-06 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173202+0000", 2024-09-21T22:50:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4370000000000007e-06 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173209+0000", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7200000000000006e-06 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183946+0000", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010736193999999999 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184038+0000", 2024-09-21T22:50:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2070000000000004e-05 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184150+0000", 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112144 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40794 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8885470890000002, 2024-09-21T22:50:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011173367, 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40794 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.89e-07 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173222+0000", 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-06 2024-09-21T22:50:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173230+0000", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173237+0000", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4200000000000001e-06 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184188+0000", 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010950295000000001 2024-09-21T22:50:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184281+0000", 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3764999999999994e-05 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184392+0000", 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011095999999999999 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40795 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8885012759999995, 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011369611, 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40795 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8999999999999997e-07 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173277+0000", 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1144999999999999e-05 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173285+0000", 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5399999999999996e-06 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173308+0000", 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3237000000000001e-05 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184427+0000", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011118303 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184521+0000", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4660999999999994e-05 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184634+0000", 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113235 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40796 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.888437165, 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011518128000000001, 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40796 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173330+0000", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173334+0000", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1200000000000004e-06 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173343+0000", 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6369999999999994e-06 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173351+0000", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.221e-06 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184666+0000", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011315733999999999 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184758+0000", 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1173000000000003e-05 2024-09-21T22:50:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184847+0000", 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9566000000000003e-05 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40797 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8884059329999996, 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011696465, 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40797 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173361+0000", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1200000000000003e-07 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173364+0000", 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173372+0000", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2369999999999992e-06 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173378+0000", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8860000000000001e-06 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184880+0000", 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011502603 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184971+0000", 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0486000000000001e-05 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185057+0000", 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5853999999999995e-05 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40798 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8883795650000001, 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011876305, 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40798 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7800000000000002e-07 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173389+0000", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173397+0000", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.841e-06 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173403+0000", 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:50:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185089+0000", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011685600000000001 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185179+0000", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0142999999999994e-05 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185263+0000", 2024-09-21T22:50:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4204999999999998e-05 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40997 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 247808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8870886120000003, 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010222111000000001, 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40997 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1399999999999998e-07 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174680+0000", 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8369999999999999e-06 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174684+0000", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6970000000000001e-06 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174689+0000", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3039999999999998e-06 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184727+0000", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010038038000000001 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184816+0000", 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9507999999999998e-05 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184900+0000", 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3412999999999996e-05 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40998 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8870730929999997, 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01102735, 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40998 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7000000000000001e-07 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174700+0000", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3999999999999997e-06 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174705+0000", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0810000000000001e-06 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174714+0000", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1169999999999993e-06 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185509+0000", 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010795157 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185617+0000", 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010754100000000001 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185720+0000", 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010378399999999999 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41173 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8839203070000003, 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010408699, 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41173 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1900000000000003e-07 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177849+0000", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.965e-06 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177858+0000", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0229999999999998e-06 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177875+0000", 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6489000000000001e-05 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187860+0000", 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099856619999999993 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187967+0000", 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106217 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188255+0000", 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000287824 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41174 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8838800610000002, 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010877736000000001, 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41174 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177887+0000", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999995e-07 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177890+0000", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.507e-06 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177900+0000", 2024-09-21T22:50:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8130000000000007e-06 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177907+0000", 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800000000000003e-06 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188332+0000", 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010425419999999999 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188450+0000", 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117992 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188764+0000", 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031363700000000002 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:52.161 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:52.162 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:52.162 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:52.163 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:52.163 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:52.164 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:52.160+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:52.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:52.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3111 trunc from 0x172f0c1 to 0xdb3c8a6 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3112 read 0x338aed7 thru 0x3399652 (0xe77c bytes) 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3113 read 0xced7a79 thru 0xcedaa6e (0x2ff6 bytes) 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3116 read 0xb0872b6 thru 0xb08b1cd (0x3f18 bytes) 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xedce728 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3118 trunc from 0xdb3c8a6 to 0xedce728 2024-09-21T22:50:52.166 INFO:teuthology.orchestra.run.smithi094.stdout:3119 write 0xd7cf8ba thru 0xd7da63c (0xad83 bytes) 2024-09-21T22:50:52.178 INFO:teuthology.orchestra.run.smithi094.stdout:3120 read 0xabfc2b thru 0xac164f (0x1a25 bytes) 2024-09-21T22:50:52.179 INFO:teuthology.orchestra.run.smithi094.stdout:3124 read 0x9253a8c thru 0x925ef7b (0xb4f0 bytes) 2024-09-21T22:50:52.179 INFO:teuthology.orchestra.run.smithi094.stdout:3125 read 0x2cce29b thru 0x2cd613a (0x7ea0 bytes) 2024-09-21T22:50:52.180 INFO:teuthology.orchestra.run.smithi094.stdout:3126 write 0xe512184 thru 0xe519968 (0x77e5 bytes) 2024-09-21T22:50:52.192 INFO:teuthology.orchestra.run.smithi094.stdout:3127 punch from 0xf7b7db to 0xf7f09a, (0x38bf bytes) 2024-09-21T22:50:52.193 INFO:teuthology.orchestra.run.smithi094.stdout:3128 write 0xddb7e6 thru 0xdea463 (0xec7e bytes) 2024-09-21T22:50:52.263 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:52.260+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:52.340 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:52.340 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:52.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:52.356 INFO:teuthology.orchestra.run.smithi094.stdout:3129 read 0x2c752f7 thru 0x2c81538 (0xc242 bytes) 2024-09-21T22:50:52.357 INFO:teuthology.orchestra.run.smithi094.stdout:3130 read 0xcdef75 thru 0xce333f (0x43cb bytes) 2024-09-21T22:50:52.358 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:52.365 INFO:teuthology.orchestra.run.smithi094.stdout:3133 trunc from 0xedce728 to 0x8fd503d 2024-09-21T22:50:52.365 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:52.360+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:52.400 INFO:teuthology.orchestra.run.smithi094.stdout:3134 writesame 0xdf3eed9 thru 0xdf4199d data_size 0x1(0x2ac5 bytes) 2024-09-21T22:50:52.400 INFO:teuthology.orchestra.run.smithi094.stdout:3135 read 0xd9b3159 thru 0xd9b58a2 (0x274a bytes) 2024-09-21T22:50:52.401 INFO:teuthology.orchestra.run.smithi094.stdout:3136 read 0x5822c17 thru 0x5832b8b (0xff75 bytes) 2024-09-21T22:50:52.402 INFO:teuthology.orchestra.run.smithi094.stdout:3139 trunc from 0xdf4199e to 0xcdd8ef8 2024-09-21T22:50:52.408 INFO:teuthology.orchestra.run.smithi094.stdout:3140 read 0xb1bf47 thru 0xb2a739 (0xe7f3 bytes) 2024-09-21T22:50:52.412 INFO:teuthology.orchestra.run.smithi094.stdout:3141 writesame 0xa6297f4 thru 0xa637cf7 data_size 0x2(0xe504 bytes) 2024-09-21T22:50:52.456 INFO:teuthology.orchestra.run.smithi094.stdout:3142 trunc from 0xcdd8ef8 to 0x15967fa 2024-09-21T22:50:52.464 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:52.460+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:52.483 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:52.483 INFO:teuthology.orchestra.run.smithi094.stdout:3149 punch from 0x51f346 to 0x529785, (0xa43f bytes) 2024-09-21T22:50:52.498 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:52.498 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:52.502 INFO:teuthology.orchestra.run.smithi094.stdout:3150 write 0x8d22136 thru 0x8d32121 (0xffec bytes) 2024-09-21T22:50:52.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:52.532 INFO:teuthology.orchestra.run.smithi094.stdout:3151 trunc from 0x8d32122 to 0x9793715 2024-09-21T22:50:52.554 INFO:teuthology.orchestra.run.smithi094.stdout:3152 write 0xd036185 thru 0xd03b812 (0x568e bytes) 2024-09-21T22:50:52.565 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:52.560+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:52.571 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:52.571 INFO:teuthology.orchestra.run.smithi094.stdout:3154 read 0x92cb30a thru 0x92d6bd2 (0xb8c9 bytes) 2024-09-21T22:50:52.579 INFO:teuthology.orchestra.run.smithi094.stdout:3155 write 0xd4eefa1 thru 0xd4f1986 (0x29e6 bytes) 2024-09-21T22:50:52.587 INFO:teuthology.orchestra.run.smithi094.stdout:3156 clone 17 order 18 su 65536 sc 7 2024-09-21T22:50:52.646 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:52.646 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:52.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:52.665 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:52.664+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:52.766 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:52.764+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:52.812 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:52.812 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:52.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:52.867 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:52.864+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:52.946 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:52.946 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:52.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:52.967 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:52.964+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:53.068 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:53.064+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:53.077 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.077 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:53.110 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.1f pg_upmap mapping to [3,5] 2024-09-21T22:50:53.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:50:53.180 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:53.176+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:53.218 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.218 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:53.269 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:53.268+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:53.370 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:53.364+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:53.390 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.390 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:53.492 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:53.488+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:53.560 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.560 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.571 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:53.568+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:53.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:53.672 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:53.664+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:53.735 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.735 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.749 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:53.773 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:53.768+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:53.874 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:53.872+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:53.915 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:53.915 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:53.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:53.976 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:53.972+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:54.045 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:54.045 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:54.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:54.105 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:54.104+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:54.124 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:50:54.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:54.191 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:54.191 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:54.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:54.212 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:54.208+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:54.278 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:54.272+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:54.373 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:54.373 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:54.380 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:54.372+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:54.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:54.480 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:54.476+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-21T22:50:54.484+0000 7f715b7fe640 0 --2- 172.21.15.94:0/1497711321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c073000 0x7f715c139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T22:50:54.512 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:54.512 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:54.513 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:54.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:54.547 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:54.570 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:54.580 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:54.576+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:54.594 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:54.594 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:54.618 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:54.639 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:54.645 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:54.645 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:54.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:54.660 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:54.660 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:54.681 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:54.682 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:54.705 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:54.704+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:54.782 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:54.782 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:54.783 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:54.780+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:54.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:54.883 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:54.876+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:54.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.567671461, 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.567628031000002, 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.968 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:54.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.567519439000002, 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.972 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.976 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.558157853000001, 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35808 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 759808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6670305120000002, 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035722402, 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35808 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292991+0000", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3769999999999999e-06 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292998+0000", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2359999999999998e-06 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293023+0000", 2024-09-21T22:50:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4765000000000001e-05 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293038+0000", 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5257e-05 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328503+0000", 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035465093000000003 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328609+0000", 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010592899999999999 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328712+0000", 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010274500000000001 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35809 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 804864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6669604719999995, 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035887960000000003, 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293061+0000", 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293071+0000", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0530000000000001e-05 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293094+0000", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3224999999999999e-05 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293109+0000", 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4646e-05 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328750+0000", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035641391000000001 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328851+0000", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026099999999999 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328947+0000", 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6617e-05 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35815 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6668914150000003, 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036042883999999997, 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35815 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293130+0000", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293135+0000", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5600000000000001e-06 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293158+0000", 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2619000000000001e-05 2024-09-21T22:50:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293172+0000", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4549000000000001e-05 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328988+0000", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035815391000000002 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329085+0000", 2024-09-21T22:50:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7112000000000001e-05 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329171+0000", 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6349999999999998e-05 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35816 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 796672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:54.994 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6668284579999995, 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036185001000000001, 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35816 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:54.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293193+0000", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2559999999999999e-06 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293199+0000", 2024-09-21T22:50:54.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293225+0000", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6213000000000001e-05 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293244+0000", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9338999999999999e-05 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329217+0000", 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035972956 2024-09-21T22:50:54.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329300+0000", 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3010000000000007e-05 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329376+0000", 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6056999999999994e-05 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.998 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35817 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 841728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6667547280000008, 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036292586000000002, 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35817 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:54.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293266+0000", 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293289+0000", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2283e-05 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293335+0000", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6597000000000001e-05 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293351+0000", 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329406+0000", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036055023999999998 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329484+0000", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7039000000000006e-05 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329558+0000", 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4233999999999994e-05 2024-09-21T22:50:55.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35824 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6666427630000005, 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03636607, 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35824 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293378+0000", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2359999999999999e-06 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293386+0000", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293408+0000", 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2713000000000001e-05 2024-09-21T22:50:55.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293424+0000", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5502000000000002e-05 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329588+0000", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036164302000000002 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329674+0000", 2024-09-21T22:50:55.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5447999999999999e-05 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329743+0000", 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9663e-05 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35825 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 833536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6665760330000001, 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036463540000000003, 2024-09-21T22:50:55.007 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35825 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293445+0000", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1230000000000001e-06 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293452+0000", 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4259999999999997e-06 2024-09-21T22:50:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293478+0000", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5423e-05 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293493+0000", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5262000000000001e-05 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329773+0000", 2024-09-21T22:50:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036279726999999998 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329847+0000", 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3842000000000002e-05 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329907+0000", 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0736999999999998e-05 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35833 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6664817680000006, 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036522798000000002, 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35833 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293539+0000", 2024-09-21T22:50:55.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293547+0000", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.14e-06 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293572+0000", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4638999999999999e-05 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293587+0000", 2024-09-21T22:50:55.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5362e-05 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329935+0000", 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036347154999999999 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330007+0000", 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2322000000000003e-05 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330061+0000", 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3953000000000003e-05 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35840 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 561152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.666412051, 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036597440000000002, 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35840 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293609+0000", 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.15e-06 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293615+0000", 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0399999999999998e-06 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293708+0000", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2972000000000004e-05 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293768+0000", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9744000000000003e-05 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330088+0000", 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036320676000000003 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330135+0000", 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6113999999999999e-05 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330205+0000", 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0743999999999999e-05 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35842 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6661808419999993, 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036523119999999999, 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35842 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293841+0000", 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.474e-06 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293849+0000", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6729999999999998e-06 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293871+0000", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2291999999999999e-05 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293888+0000", 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512999999999999e-05 2024-09-21T22:50:55.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330234+0000", 2024-09-21T22:50:55.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036346373000000001 2024-09-21T22:50:55.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330308+0000", 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3770999999999996e-05 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330362+0000", 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4024000000000003e-05 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35848 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 552960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6661103080000004, 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036565919000000002, 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35848 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293911+0000", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1570000000000001e-06 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293916+0000", 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294560+0000", 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00064339699999999998 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294578+0000", 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8318999999999999e-05 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330391+0000", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035812716000000001 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330434+0000", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3421000000000003e-05 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330475+0000", 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1242999999999998e-05 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35849 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 598016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6654109080000001, 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036033295999999999, 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35849 2024-09-21T22:50:55.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294611+0000", 2024-09-21T22:50:55.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-06 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294620+0000", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1300000000000007e-06 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294643+0000", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2849000000000001e-05 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294660+0000", 2024-09-21T22:50:55.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7419e-05 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330504+0000", 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035843815000000001 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330571+0000", 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7446000000000004e-05 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330642+0000", 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0900000000000002e-05 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35850 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 643072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6653391010000007, 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036113026999999999, 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35850 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294682+0000", 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2240000000000001e-06 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294688+0000", 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0530000000000003e-06 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294725+0000", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807999999999997e-05 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295530+0000", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080478500000000003 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330672+0000", 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035142709000000001 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330748+0000", 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5794e-05 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330794+0000", 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5654e-05 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35857 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6643847320000003, 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035441063000000002, 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35857 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:55.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295636+0000", 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295642+0000", 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295662+0000", 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9775e-05 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295677+0000", 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4606e-05 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330937+0000", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035260540999999999 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330998+0000", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0763000000000001e-05 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331076+0000", 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8197999999999999e-05 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35858 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6643242249999997, 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035527577999999997, 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35858 2024-09-21T22:50:55.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295697+0000", 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295703+0000", 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.993e-06 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295740+0000", 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6984999999999999e-05 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.296513+0000", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077355700000000002 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331106+0000", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034593169999999999 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331179+0000", 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2705000000000003e-05 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331223+0000", 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4091000000000002e-05 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:49450 3.1d 3:bc968670:::rbd_data.111e5668b648.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1330961659999996, 2024-09-21T22:50:55.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041268020000000002, 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 49450 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826926+0000", 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834639+0000", 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077131559999999997 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834665+0000", 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6054000000000002e-05 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834681+0000", 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6161e-05 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834780+0000", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8319999999999994e-05 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834942+0000", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016233 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.836568+0000", 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00162598 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.867947+0000", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031378915 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868057+0000", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109911 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868128+0000", 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1236999999999996e-05 2024-09-21T22:50:55.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868192+0000", 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3806000000000005e-05 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.054 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:55.056 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:55.048+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:55.157 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:55.152+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:55.184 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:55.184 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:55.258 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:55.252+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:55.321 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:55.321 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:55.321 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:55.321 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:55.321 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:50:55.358 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:55.356+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:55.454 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:55.454 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.516173244999997, 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:55.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:50:55.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:55.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9711657700000007, 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:50:55.463 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:50:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:50:55.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9711013860000008, 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:50:55.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9710379360000001, 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:50:55.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9709747580000005, 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:50:55.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9709184180000001, 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:50:55.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:50:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:55.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9708766759999996, 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:50:55.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:55.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9708160750000001, 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9707356390000008, 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:50:55.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:50:55.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:55.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9706595290000006, 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:50:55.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:50:55.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9705724559999993, 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:50:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.970528968, 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:50:55.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9704953300000003, 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:50:55.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9704615640000007, 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:50:55.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9704302800000004, 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:50:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9703849249999994, 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9703534289999993, 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:50:55.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.551 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9703186479999992, 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9523731170000005, 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:50:55.557 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:50:55.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9523120729999999, 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:55.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:50:55.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:55.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:50:55.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:55.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:55.568 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:50:55.570 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:55.564+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:55.592 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone16 from 0xd4f1987 (overlap 0x15967fa) to 0xf1c8c8 2024-09-21T22:50:55.670 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:55.664+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:55.753 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:55.753 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.768 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:50:55.770 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:55.764+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:55.790 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T22:50:55.871 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:55.864+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:55.892 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:55.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:55.892 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:55.892 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:55.892 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:55.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:50:55.972 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:55.968+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.013543567999999, 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:50:56.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:50:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.009305998999999, 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:56.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:50:56.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:50:56.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:56.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:50:56.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.009111100999998, 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:50:56.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:50:56.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:50:56.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34552 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.865629616, 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013168394, 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34552 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:50:56.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167254+0000", 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2320000000000003e-06 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167273+0000", 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9502999999999999e-05 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167323+0000", 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9527999999999997e-05 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167346+0000", 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3618999999999999e-05 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167513+0000", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166992 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167758+0000", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000244622 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.169268+0000", 2024-09-21T22:50:56.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015101559999999999 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180086+0000", 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010817937999999999 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180137+0000", 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0633999999999997e-05 2024-09-21T22:50:56.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180295+0000", 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015844899999999999 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180418+0000", 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012272100000000001 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34557 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.860686854000001, 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010897125000000001, 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34557 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172195+0000", 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7099999999999999e-06 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172210+0000", 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4823e-05 2024-09-21T22:50:56.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172240+0000", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0361999999999998e-05 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172255+0000", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552000000000001e-05 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172390+0000", 2024-09-21T22:50:56.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013588399999999999 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172680+0000", 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028962999999999998 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.174751+0000", 2024-09-21T22:50:56.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020706689999999998 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182868+0000", 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081170249999999999 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182936+0000", 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8422000000000003e-05 2024-09-21T22:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183021+0000", 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5043000000000006e-05 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183089+0000", 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8004999999999997e-05 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40788 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 190464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8598935389999998, 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012747681, 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40788 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172986+0000", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5499999999999998e-07 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172990+0000", 2024-09-21T22:50:56.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5820000000000002e-06 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172998+0000", 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7679999999999997e-06 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173022+0000", 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3747000000000002e-05 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182235+0000", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092129110000000007 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182364+0000", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128589 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185733+0000", 2024-09-21T22:50:56.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003369529 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40789 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8598467599999999, 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010153383, 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40789 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:56.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173033+0000", 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6400000000000003e-07 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173035+0000", 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8389999999999998e-06 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173044+0000", 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173049+0000", 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5330000000000002e-06 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182955+0000", 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099058489999999996 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183053+0000", 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8072000000000006e-05 2024-09-21T22:50:56.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183186+0000", 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013232700000000001 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40790 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8598212840000006, 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010372809, 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40790 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7399999999999998e-07 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173061+0000", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.853e-06 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173069+0000", 2024-09-21T22:50:56.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3340000000000002e-06 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173077+0000", 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9780000000000005e-06 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183219+0000", 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010142004 2024-09-21T22:50:56.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183319+0000", 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9908000000000006e-05 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183430+0000", 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111258 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40791 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8597911200000006, 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010578489, 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40791 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6899999999999998e-07 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173110+0000", 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1763000000000002e-05 2024-09-21T22:50:56.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173120+0000", 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8740000000000007e-06 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173135+0000", 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5356999999999999e-05 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183465+0000", 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010329823 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183558+0000", 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2762000000000004e-05 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183666+0000", 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108441 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40792 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8597295410000001, 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01076204, 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40792 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173150+0000", 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173163+0000", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3438000000000001e-05 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173173+0000", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5689999999999994e-06 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173180+0000", 2024-09-21T22:50:56.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7889999999999997e-06 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183703+0000", 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010523341 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183800+0000", 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6836999999999995e-05 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183911+0000", 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011145300000000001 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40793 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 223232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8596887730000002, 2024-09-21T22:50:56.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010959662, 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40793 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173191+0000", 2024-09-21T22:50:56.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1399999999999997e-07 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173193+0000", 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5830000000000001e-06 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173202+0000", 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4370000000000007e-06 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173209+0000", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7200000000000006e-06 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183946+0000", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010736193999999999 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184038+0000", 2024-09-21T22:50:56.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2070000000000004e-05 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184150+0000", 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112144 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40794 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8596598859999993, 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011173367, 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40794 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.89e-07 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173222+0000", 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-06 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173230+0000", 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173237+0000", 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4200000000000001e-06 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184188+0000", 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010950295000000001 2024-09-21T22:50:56.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184281+0000", 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3764999999999994e-05 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184392+0000", 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011095999999999999 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40795 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8596140729999995, 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011369611, 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40795 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8999999999999997e-07 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173277+0000", 2024-09-21T22:50:56.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1144999999999999e-05 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173285+0000", 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5399999999999996e-06 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173308+0000", 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3237000000000001e-05 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184427+0000", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011118303 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184521+0000", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4660999999999994e-05 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184634+0000", 2024-09-21T22:50:56.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113235 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40796 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8595499619999991, 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011518128000000001, 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40796 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173330+0000", 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173334+0000", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1200000000000004e-06 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173343+0000", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6369999999999994e-06 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173351+0000", 2024-09-21T22:50:56.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.221e-06 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184666+0000", 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011315733999999999 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184758+0000", 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1173000000000003e-05 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184847+0000", 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9566000000000003e-05 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40797 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8595187299999996, 2024-09-21T22:50:56.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011696465, 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40797 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:56.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173361+0000", 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1200000000000003e-07 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173364+0000", 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173372+0000", 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2369999999999992e-06 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173378+0000", 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8860000000000001e-06 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184880+0000", 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011502603 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184971+0000", 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0486000000000001e-05 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185057+0000", 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5853999999999995e-05 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40798 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8594923619999992, 2024-09-21T22:50:56.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011876305, 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40798 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7800000000000002e-07 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173389+0000", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173397+0000", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.841e-06 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173403+0000", 2024-09-21T22:50:56.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185089+0000", 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011685600000000001 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185179+0000", 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0142999999999994e-05 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185263+0000", 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4204999999999998e-05 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40997 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 247808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:56.134 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8582014089999994, 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010222111000000001, 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40997 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1399999999999998e-07 2024-09-21T22:50:56.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174680+0000", 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8369999999999999e-06 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174684+0000", 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6970000000000001e-06 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174689+0000", 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3039999999999998e-06 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184727+0000", 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010038038000000001 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184816+0000", 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9507999999999998e-05 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184900+0000", 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3412999999999996e-05 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40998 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8581858899999997, 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01102735, 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40998 2024-09-21T22:50:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7000000000000001e-07 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174700+0000", 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3999999999999997e-06 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174705+0000", 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0810000000000001e-06 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174714+0000", 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1169999999999993e-06 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185509+0000", 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010795157 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185617+0000", 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010754100000000001 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185720+0000", 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010378399999999999 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41173 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8550331040000003, 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010408699, 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41173 2024-09-21T22:50:56.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1900000000000003e-07 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177849+0000", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.965e-06 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177858+0000", 2024-09-21T22:50:56.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0229999999999998e-06 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177875+0000", 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6489000000000001e-05 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187860+0000", 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099856619999999993 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187967+0000", 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106217 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188255+0000", 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000287824 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41174 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8549928579999992, 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010877736000000001, 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41174 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177887+0000", 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999995e-07 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177890+0000", 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.507e-06 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177900+0000", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8130000000000007e-06 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177907+0000", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800000000000003e-06 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188332+0000", 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010425419999999999 2024-09-21T22:50:56.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188450+0000", 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117992 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188764+0000", 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031363700000000002 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:56.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:56.155 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:56.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:50:56.157 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:56.152+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:56.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:50:56.257 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:56.252+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:56.280 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:56.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:56.280 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:56.281 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:56.282 INFO:teuthology.orchestra.run.smithi094.stdout:3157 write 0xb13726 thru 0xb14bd0 (0x14ab bytes) 2024-09-21T22:50:56.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:50:56.357 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:56.356+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:56.455 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:56.455 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:56.455 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:50:56.455 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:50:56.455 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:56.458 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:56.452+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:56.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:50:56.560 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:56.556+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.149896614999999, 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:50:56.631 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:50:56.635 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.149856985, 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:50:56.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:50:56.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.639 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.149091500000001, 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:50:56.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.148877552, 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:50:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:50:56.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.148787559000001, 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:50:56.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:50:56.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.148726343, 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:50:56.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878849729999995, 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:50:56.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:50:56.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878714259999995, 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:50:56.662 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:50:56.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:50:56.666 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878510440000007, 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:50:56.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:50:56.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878376149999998, 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.671 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:50:56.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:50:56.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878231520000003, 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:50:56.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:50:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5878105270000002, 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:50:56.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5877721270000009, 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:50:56.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5877476549999994, 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:50:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:50:56.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5877287899999999, 2024-09-21T22:50:56.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:50:56.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:50:56.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.696 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5877155530000007, 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:50:56.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:50:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5877025800000002, 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.701 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:50:56.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:50:56.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5876897860000003, 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:50:56.705 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:50:56.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:50:56.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.709 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5876772849999998, 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:50:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:50:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.5873668589999994, 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:50:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:50:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:50:56.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.718 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:50:56.718 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:50:56.718 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:56.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:50:56.720 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:50:56.720 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:56.716+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:56.820 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:56.816+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:56.894 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:50:56.894 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:50:56.894 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:50:56.894 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:50:56.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:50:56.910 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:56.910 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:56.926 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:56.927 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:56.920+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:56.943 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:56.943 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:56.960 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:56.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:57.001 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:57.001 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:57.021 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:57.016+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:57.022 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.022 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.022 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:57.022 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:57.029 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:50:57.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:50:57.052 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:57.052 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:57.063 INFO:teuthology.orchestra.run.smithi094.stdout:3158 trunc from 0xd4f1987 to 0xcf2f99e 2024-09-21T22:50:57.097 INFO:teuthology.orchestra.run.smithi094.stdout:3160 trunc from 0xcf2f99e to 0x63e4b74 2024-09-21T22:50:57.122 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:57.116+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:57.128 INFO:teuthology.orchestra.run.smithi094.stdout:3161 read 0x1ce4bef thru 0x1ceb5c0 (0x69d2 bytes) 2024-09-21T22:50:57.128 INFO:teuthology.orchestra.run.smithi094.stdout:3162 punch from 0x1644304 to 0x164ce11, (0x8b0d bytes) 2024-09-21T22:50:57.128 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.128 INFO:teuthology.orchestra.run.smithi094.stdout:3166 read 0x1d0c478 thru 0x1d16e1a (0xa9a3 bytes) 2024-09-21T22:50:57.145 INFO:teuthology.orchestra.run.smithi094.stdout:3167 writesame 0xcd48916 thru 0xcd4ad77 data_size 0x1(0x2462 bytes) 2024-09-21T22:50:57.153 INFO:teuthology.orchestra.run.smithi094.stdout:3168 write 0xcf3977d thru 0xcf45f37 (0xc7bb bytes) 2024-09-21T22:50:57.160 INFO:teuthology.orchestra.run.smithi094.stdout:3170 read 0xb627d5b thru 0xb62984a (0x1af0 bytes) 2024-09-21T22:50:57.160 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.160 INFO:teuthology.orchestra.run.smithi094.stdout:3173 writesame 0x56566d2 thru 0x565cf38 data_size 0x3b(0x6867 bytes) 2024-09-21T22:50:57.167 INFO:teuthology.orchestra.run.smithi094.stdout:3174 punch from 0x5f5ff10 to 0x5f6c747, (0xc837 bytes) 2024-09-21T22:50:57.167 INFO:teuthology.orchestra.run.smithi094.stdout:3176 writesame 0x491e0ae thru 0x492c5e0 data_size 0x5(0xe533 bytes) 2024-09-21T22:50:57.183 INFO:teuthology.orchestra.run.smithi094.stdout:3177 punch from 0x4803a04 to 0x48061d6, (0x27d2 bytes) 2024-09-21T22:50:57.183 INFO:teuthology.orchestra.run.smithi094.stdout:3178 writesame 0x6523a7d thru 0x6524e2e data_size 0x1(0x13b2 bytes) 2024-09-21T22:50:57.185 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.185 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.188 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.188 INFO:teuthology.orchestra.run.smithi094.stdout:3180 punch from 0xf0a162 to 0xf15a07, (0xb8a5 bytes) 2024-09-21T22:50:57.188 INFO:teuthology.orchestra.run.smithi094.stdout:3183 punch from 0x649593b to 0x6495aee, (0x1b3 bytes) 2024-09-21T22:50:57.188 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:50:57.202 INFO:teuthology.orchestra.run.smithi094.stdout:3185 write 0xd962b1d thru 0xd9640a4 (0x1588 bytes) 2024-09-21T22:50:57.210 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.210 INFO:teuthology.orchestra.run.smithi094.stdout:3187 read 0x3528c46 thru 0x35386e8 (0xfaa3 bytes) 2024-09-21T22:50:57.210 INFO:teuthology.orchestra.run.smithi094.stdout:3188 writesame 0x366031d thru 0x3663a34 data_size 0x8(0x3718 bytes) 2024-09-21T22:50:57.216 INFO:teuthology.orchestra.run.smithi094.stdout:3189 write 0x4d3fae8 thru 0x4d417e8 (0x1d01 bytes) 2024-09-21T22:50:57.223 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:57.220+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:57.225 INFO:teuthology.orchestra.run.smithi094.stdout:3191 writesame 0x7ee6ed4 thru 0x7eeeb65 data_size 0xf(0x7c92 bytes) 2024-09-21T22:50:57.233 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.233 INFO:teuthology.orchestra.run.smithi094.stdout:3193 punch from 0xcef6302 to 0xcef9ee6, (0x3be4 bytes) 2024-09-21T22:50:57.253 INFO:teuthology.orchestra.run.smithi094.stdout:3194 trunc from 0xd9640a5 to 0x3621d16 2024-09-21T22:50:57.285 INFO:teuthology.orchestra.run.smithi094.stdout:3195 read 0x2d77e29 thru 0x2d7da0e (0x5be6 bytes) 2024-09-21T22:50:57.299 INFO:teuthology.orchestra.run.smithi094.stdout:3198 write 0x6d01c5f thru 0x6d0f1e4 (0xd586 bytes) 2024-09-21T22:50:57.307 INFO:teuthology.orchestra.run.smithi094.stdout:3199 read 0x3b4fbbc thru 0x3b4fdec (0x231 bytes) 2024-09-21T22:50:57.315 INFO:teuthology.orchestra.run.smithi094.stdout:3200 trunc from 0x6d0f1e5 to 0x1e72638 2024-09-21T22:50:57.323 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:57.320+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:57.354 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.354 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:50:57.380 INFO:teuthology.orchestra.run.smithi094.stdout:3202 write 0xa7df8d1 thru 0xa7ebc45 (0xc375 bytes) 2024-09-21T22:50:57.386 INFO:teuthology.orchestra.run.smithi094.stdout:3203 writesame 0x6e52df8 thru 0x6e5db2f data_size 0x5c(0xad38 bytes) 2024-09-21T22:50:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:3204 write 0xc47f1cb thru 0xc48d040 (0xde76 bytes) 2024-09-21T22:50:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:3206 write 0xd447269 thru 0xd452842 (0xb5da bytes) 2024-09-21T22:50:57.424 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:50:57.420+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:50:57.432 INFO:teuthology.orchestra.run.smithi094.stdout:3207 write 0x5107757 thru 0x51119d8 (0xa282 bytes) 2024-09-21T22:50:57.441 INFO:teuthology.orchestra.run.smithi094.stdout:3208 punch from 0xc82aed8 to 0xc82f2d9, (0x4401 bytes) 2024-09-21T22:50:57.441 INFO:teuthology.orchestra.run.smithi094.stdout:3209 write 0x537da42 thru 0x5383562 (0x5b21 bytes) 2024-09-21T22:50:57.450 INFO:teuthology.orchestra.run.smithi094.stdout:3210 punch from 0x123f5e9 to 0x12443b1, (0x4dc8 bytes) 2024-09-21T22:50:57.450 INFO:teuthology.orchestra.run.smithi094.stdout:3213 read 0x2cdf5e2 thru 0x2ce2957 (0x3376 bytes) 2024-09-21T22:50:57.450 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.450 INFO:teuthology.orchestra.run.smithi094.stdout:3216 write 0x98c36d4 thru 0x98d17f5 (0xe122 bytes) 2024-09-21T22:50:57.481 INFO:teuthology.orchestra.run.smithi094.stdout:3217 trunc from 0xd452843 to 0x32679c3 2024-09-21T22:50:57.515 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.515 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.516 INFO:teuthology.orchestra.run.smithi094.stdout:3218 trunc from 0x32679c3 to 0x7795116 2024-09-21T22:50:57.525 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:57.520+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:57.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:50:57.544 INFO:teuthology.orchestra.run.smithi094.stdout:3219 write 0x9c8863a thru 0x9c9401b (0xb9e2 bytes) 2024-09-21T22:50:57.566 INFO:teuthology.orchestra.run.smithi094.stdout:3220 write 0xc44bc05 thru 0xc458216 (0xc612 bytes) 2024-09-21T22:50:57.587 INFO:teuthology.orchestra.run.smithi094.stdout:3221 write 0xebd5217 thru 0xebddfa5 (0x8d8f bytes) 2024-09-21T22:50:57.594 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.623 INFO:teuthology.orchestra.run.smithi094.stdout:3223 trunc from 0xebddfa6 to 0x50f6dd 2024-09-21T22:50:57.626 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:57.620+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:57.672 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.672 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:50:57.732 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:57.728+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:57.775 INFO:teuthology.orchestra.run.smithi094.stdout:3224 trunc from 0x50f6dd to 0x21387c2 2024-09-21T22:50:57.783 INFO:teuthology.orchestra.run.smithi094.stdout:3226 read 0x10ac23c thru 0x10bacdd (0xeaa2 bytes) 2024-09-21T22:50:57.783 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.783 INFO:teuthology.orchestra.run.smithi094.stdout:3229 read 0x1974706 thru 0x19792e7 (0x4be2 bytes) 2024-09-21T22:50:57.783 INFO:teuthology.orchestra.run.smithi094.stdout:3230 read 0x112604f thru 0x112903a (0x2fec bytes) 2024-09-21T22:50:57.784 INFO:teuthology.orchestra.run.smithi094.stdout:3231 write 0x3e3112 thru 0x3f0ae0 (0xd9cf bytes) 2024-09-21T22:50:57.828 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:57.820+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:3232 writesame 0x5a7898c thru 0x5a8443b data_size 0x1d(0xbab0 bytes) 2024-09-21T22:50:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:3233 write 0x4f6aabe thru 0x4f7984b (0xed8e bytes) 2024-09-21T22:50:57.852 INFO:teuthology.orchestra.run.smithi094.stdout:3234 write 0x5f43a2d thru 0x5f48726 (0x4cfa bytes) 2024-09-21T22:50:57.874 INFO:teuthology.orchestra.run.smithi094.stdout:3235 writesame 0x71139b5 thru 0x7113df5 data_size 0x3(0x441 bytes) 2024-09-21T22:50:57.874 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.874 INFO:teuthology.orchestra.run.smithi094.stdout:3237 punch from 0x22ab1cd to 0x22b0a40, (0x5873 bytes) 2024-09-21T22:50:57.874 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:57.874 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:57.875 INFO:teuthology.orchestra.run.smithi094.stdout:3239 punch from 0x6458877 to 0x6461c16, (0x939f bytes) 2024-09-21T22:50:57.875 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.875 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:57.875 INFO:teuthology.orchestra.run.smithi094.stdout:3244 clone 18 order 18 su 1024 sc 12 2024-09-21T22:50:57.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:50:57.929 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:57.928+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:58.030 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:58.024+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:58.048 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:58.049 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:58.063 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:50:58.130 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:58.128+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:58.133 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:50:58.134 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:50:58.134 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.8350566713471808 2024-09-21T22:50:58.134 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.8350566713471808 2024-09-21T22:50:58.231 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:58.228+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:58.232 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:58.233 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:58.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:50:58.280 INFO:teuthology.orchestra.run.smithi094.stdout:flattening image image_client.0-clone17 2024-09-21T22:50:58.331 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:58.328+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:58.395 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:58.395 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:58.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:50:58.666 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:58.660+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:58.748 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T22:50:58.768 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:58.764+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:58.818 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:58.818 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:58.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:50:58.882 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:50:58.880+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:50:58.967 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:50:58.964+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:50:59.023 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:59.024 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:59.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:50:59.054 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:50:59.074 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:50:59.085 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:59.084+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 0.835057 (d5c6) 2024-09-21T22:50:59.169 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:59.164+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:59.222 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:50:59.222 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:50:59.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:50:59.251 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:50:59.251 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:50:59.268 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:50:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:50:59.269 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:50:59.264+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:50:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:50:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:50:59.304 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:50:59.304 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:50:59.321 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:50:59.321 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:50:59.342 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:50:59.342 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:50:59.360 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:50:59.360 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:50:59.370 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:59.364+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:59.375 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:59.375 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:59.376 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:59.376 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:59.376 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:59.379 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:50:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:50:59.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:50:59.425 INFO:teuthology.orchestra.run.smithi094.stdout:3247 read 0x17d928f thru 0x17e3c79 (0xa9eb bytes) 2024-09-21T22:50:59.430 INFO:teuthology.orchestra.run.smithi094.stdout:3249 trunc from 0x7113df6 to 0x839d865 2024-09-21T22:50:59.441 INFO:teuthology.orchestra.run.smithi094.stdout:3250 read 0x221ccd2 thru 0x222351a (0x6849 bytes) 2024-09-21T22:50:59.441 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:59.442 INFO:teuthology.orchestra.run.smithi094.stdout:3252 trunc from 0x839d865 to 0x8ac0eb3 2024-09-21T22:50:59.448 INFO:teuthology.orchestra.run.smithi094.stdout:3253 read 0x73765e6 thru 0x737da15 (0x7430 bytes) 2024-09-21T22:50:59.450 INFO:teuthology.orchestra.run.smithi094.stdout:3254 trunc from 0x8ac0eb3 to 0x9f93591 2024-09-21T22:50:59.457 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:59.457 INFO:teuthology.orchestra.run.smithi094.stdout:3257 read 0x1d72952 thru 0x1d77917 (0x4fc6 bytes) 2024-09-21T22:50:59.467 INFO:teuthology.orchestra.run.smithi094.stdout:3258 trunc from 0x9f93591 to 0x3e03c59 2024-09-21T22:50:59.471 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:50:59.468+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:50:59.493 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:59.493 INFO:teuthology.orchestra.run.smithi094.stdout:3261 read 0x363a207 thru 0x36437b8 (0x95b2 bytes) 2024-09-21T22:50:59.493 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:59.494 INFO:teuthology.orchestra.run.smithi094.stdout:3263 write 0xf34fe2 thru 0xf42cd0 (0xdcef bytes) 2024-09-21T22:50:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:3264 read 0x301c293 thru 0x302072e (0x449c bytes) 2024-09-21T22:50:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:3265 read 0x108f55b thru 0x1099867 (0xa30d bytes) 2024-09-21T22:50:59.516 INFO:teuthology.orchestra.run.smithi094.stdout:3266 trunc from 0x3e03c59 to 0x386e7a1 2024-09-21T22:50:59.530 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:50:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:50:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9332 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.134392712, 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036865955999999998, 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9332 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392348+0000", 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392349+0000", 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392352+0000", 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-21T22:50:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392361+0000", 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2730000000000008e-06 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392382+0000", 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0295000000000001e-05 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425453+0000", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033070961000000003 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.425604+0000", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015153700000000001 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.429214+0000", 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036101839999999998 2024-09-21T22:50:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9333 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.134349281999999, 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038890734000000003, 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9333 2024-09-21T22:50:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392392+0000", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392393+0000", 2024-09-21T22:50:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0130000000000001e-06 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392404+0000", 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1429e-05 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392415+0000", 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0593e-05 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392422+0000", 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5560000000000002e-06 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.430930+0000", 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038507825000000002 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431102+0000", 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017161300000000001 2024-09-21T22:50:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431283+0000", 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180705 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9334 3.1d 3:bc2a803e:::rbd_data.111e1fb363f5.0000000000000000:9 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.134240689999999, 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039705442000000001, 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9334 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392500+0000", 2024-09-21T22:50:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392502+0000", 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.57e-06 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392509+0000", 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392526+0000", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7798000000000001e-05 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.392541+0000", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4639e-05 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.431906+0000", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0393651 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432063+0000", 2024-09-21T22:50:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015709 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432206+0000", 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142752 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:9359 3.5 3:a201daf0:::rbd_data.111e33663d62.000000000000000a:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.124879104000001, 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038973980999999998, 2024-09-21T22:50:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9359 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401862+0000", 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401864+0000", 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7999999999999999e-06 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401873+0000", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3759999999999997e-06 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401895+0000", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1484999999999998e-05 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.401908+0000", 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432257+0000", 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03034912 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.432476+0000", 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218676 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:28.440836+0000", 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083598309999999999 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35808 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 759808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233751763000001, 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035722402, 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35808 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292989+0000", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292991+0000", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3769999999999999e-06 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.292998+0000", 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2359999999999998e-06 2024-09-21T22:50:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293023+0000", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4765000000000001e-05 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293038+0000", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5257e-05 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328503+0000", 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035465093000000003 2024-09-21T22:50:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328609+0000", 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010592899999999999 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328712+0000", 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010274500000000001 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35809 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 804864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233681723, 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035887960000000003, 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293059+0000", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293061+0000", 2024-09-21T22:50:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293071+0000", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0530000000000001e-05 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293094+0000", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3224999999999999e-05 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293109+0000", 2024-09-21T22:50:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4646e-05 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328750+0000", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035641391000000001 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328851+0000", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026099999999999 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328947+0000", 2024-09-21T22:50:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6617e-05 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35815 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233612666000001, 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036042883999999997, 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35815 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293128+0000", 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293130+0000", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293135+0000", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5600000000000001e-06 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293158+0000", 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2619000000000001e-05 2024-09-21T22:50:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293172+0000", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4549000000000001e-05 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.328988+0000", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035815391000000002 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329085+0000", 2024-09-21T22:50:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7112000000000001e-05 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329171+0000", 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6349999999999998e-05 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35816 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 796672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233549709, 2024-09-21T22:50:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036185001000000001, 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35816 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293191+0000", 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293193+0000", 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2559999999999999e-06 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293199+0000", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293225+0000", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6213000000000001e-05 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293244+0000", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9338999999999999e-05 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329217+0000", 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035972956 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329300+0000", 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3010000000000007e-05 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329376+0000", 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6056999999999994e-05 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35817 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 841728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233475979, 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036292586000000002, 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35817 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293265+0000", 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293266+0000", 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293289+0000", 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2283e-05 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293335+0000", 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6597000000000001e-05 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293351+0000", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329406+0000", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036055023999999998 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329484+0000", 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7039000000000006e-05 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329558+0000", 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4233999999999994e-05 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35824 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233364013999999, 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03636607, 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35824 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293377+0000", 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293378+0000", 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2359999999999999e-06 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293386+0000", 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2060000000000001e-06 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293408+0000", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2713000000000001e-05 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293424+0000", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5502000000000002e-05 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329588+0000", 2024-09-21T22:50:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036164302000000002 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329674+0000", 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5447999999999999e-05 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329743+0000", 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9663e-05 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35825 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 833536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233297284000001, 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036463540000000003, 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35825 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293444+0000", 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293445+0000", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1230000000000001e-06 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293452+0000", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4259999999999997e-06 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293478+0000", 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5423e-05 2024-09-21T22:50:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293493+0000", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5262000000000001e-05 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329773+0000", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036279726999999998 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329847+0000", 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3842000000000002e-05 2024-09-21T22:50:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329907+0000", 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0736999999999998e-05 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35833 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233203018999999, 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036522798000000002, 2024-09-21T22:50:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35833 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293538+0000", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293539+0000", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293547+0000", 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.14e-06 2024-09-21T22:50:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293572+0000", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4638999999999999e-05 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293587+0000", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5362e-05 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.329935+0000", 2024-09-21T22:50:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036347154999999999 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330007+0000", 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2322000000000003e-05 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330061+0000", 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3953000000000003e-05 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35840 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 561152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.233133302000001, 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036597440000000002, 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35840 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293608+0000", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293609+0000", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.15e-06 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293615+0000", 2024-09-21T22:50:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0399999999999998e-06 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293708+0000", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2972000000000004e-05 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293768+0000", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9744000000000003e-05 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330088+0000", 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036320676000000003 2024-09-21T22:50:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330135+0000", 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6113999999999999e-05 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330205+0000", 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0743999999999999e-05 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35842 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.232902093, 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036523119999999999, 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35842 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293839+0000", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293841+0000", 2024-09-21T22:50:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.474e-06 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293849+0000", 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6729999999999998e-06 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293871+0000", 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2291999999999999e-05 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293888+0000", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512999999999999e-05 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330234+0000", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036346373000000001 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330308+0000", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3770999999999996e-05 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330362+0000", 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4024000000000003e-05 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35848 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 552960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.232831558999999, 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036565919000000002, 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35848 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293910+0000", 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293911+0000", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1570000000000001e-06 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.293916+0000", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294560+0000", 2024-09-21T22:50:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00064339699999999998 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294578+0000", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8318999999999999e-05 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330391+0000", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035812716000000001 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330434+0000", 2024-09-21T22:50:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3421000000000003e-05 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330475+0000", 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1242999999999998e-05 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35849 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 598016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.232132159000001, 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036033295999999999, 2024-09-21T22:50:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35849 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294609+0000", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294611+0000", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-06 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294620+0000", 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1300000000000007e-06 2024-09-21T22:50:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294643+0000", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2849000000000001e-05 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294660+0000", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7419e-05 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330504+0000", 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035843815000000001 2024-09-21T22:50:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330571+0000", 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7446000000000004e-05 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330642+0000", 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0900000000000002e-05 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35850 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 643072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.232060352, 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036113026999999999, 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35850 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294681+0000", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294682+0000", 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2240000000000001e-06 2024-09-21T22:50:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294688+0000", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0530000000000003e-06 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.294725+0000", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6807999999999997e-05 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295530+0000", 2024-09-21T22:50:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080478500000000003 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330672+0000", 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035142709000000001 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330748+0000", 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5794e-05 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330794+0000", 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5654e-05 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35857 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.231105983000001, 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035441063000000002, 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35857 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295635+0000", 2024-09-21T22:50:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295636+0000", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2899999999999999e-06 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295642+0000", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295662+0000", 2024-09-21T22:50:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9775e-05 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295677+0000", 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4606e-05 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330937+0000", 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035260540999999999 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.330998+0000", 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0763000000000001e-05 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331076+0000", 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8197999999999999e-05 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:35858 3.16 3:69791289:::rbd_data.111ebb10c685.000000000000001e:e [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.231045476, 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035527577999999997, 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35858 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295696+0000", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295697+0000", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295703+0000", 2024-09-21T22:50:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.993e-06 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.295740+0000", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6984999999999999e-05 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.296513+0000", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077355700000000002 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331106+0000", 2024-09-21T22:50:59.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034593169999999999 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331179+0000", 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2705000000000003e-05 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.331223+0000", 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4091000000000002e-05 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:49450 3.1d 3:bc968670:::rbd_data.111e5668b648.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6998174170000002, 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041268020000000002, 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 49450 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:50:59.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826924+0000", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.826926+0000", 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-21T22:50:59.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834639+0000", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077131559999999997 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834665+0000", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6054000000000002e-05 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834681+0000", 2024-09-21T22:50:59.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6161e-05 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834780+0000", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8319999999999994e-05 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.834942+0000", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016233 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.836568+0000", 2024-09-21T22:50:59.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00162598 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.867947+0000", 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031378915 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868057+0000", 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109911 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:50:59.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868128+0000", 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1236999999999996e-05 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:49.868192+0000", 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3806000000000005e-05 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:50:59.620 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:59.621 INFO:teuthology.orchestra.run.smithi094.stdout:3268 read 0xedefda thru 0xee8ea1 (0x9ec8 bytes) 2024-09-21T22:50:59.622 INFO:teuthology.orchestra.run.smithi094.stdout:3269 writesame 0x58bc74 thru 0x595443 data_size 0x38(0x97d0 bytes) 2024-09-21T22:50:59.622 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:50:59.616+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:50:59.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:50:59.692 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:50:59.692 INFO:teuthology.orchestra.run.smithi094.stdout:3271 punch from 0x3669cf0 to 0x366a2db, (0x5eb bytes) 2024-09-21T22:50:59.699 INFO:teuthology.orchestra.run.smithi094.stdout:3273 writesame 0x3f668db thru 0x3f70a0a data_size 0x8(0xa130 bytes) 2024-09-21T22:50:59.699 INFO:teuthology.orchestra.run.smithi094.stdout:3274 clone 19 order 19 su 32768 sc 12 2024-09-21T22:50:59.722 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:59.716+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:59.791 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:59.791 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:59.791 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:50:59.791 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:59.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:50:59.823 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:50:59.820+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:50:59.924 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:50:59.920+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:50:59.925 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:50:59.925 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:50:59.925 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:50:59.925 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:50:59.925 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:50:59.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:00.026 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:00.020+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:00.056 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.118111725000006, 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.57310425, 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:51:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.573039866, 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572976415999999, 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572913238, 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572856898, 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572815156000001, 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572754554999999, 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572674119, 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572598009, 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572510936, 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572467447999999, 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.57243381, 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572400044, 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.57236876, 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572323405000001, 2024-09-21T22:51:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572291909, 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:00.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.572257128, 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:00.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.554311597, 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:51:00.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:51:00.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:51:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.554250552999999, 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:00.145 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:00.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:00.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:00.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:00.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.150 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:00.151 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:00.152 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:00.148+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:51:00.252 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:00.248+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:00.274 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone18 from 0x3f70a0b (overlap 0x386e7a1) to 0x1692973 2024-09-21T22:51:00.352 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:00.352 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:00.352 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:00.352 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:00.353 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:00.352+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:00.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:00.457 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:00.456+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:00.531 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T22:51:00.535 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:00.535 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:00.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:00.594 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:00.592+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:00.656 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:00.648+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:00.722 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.700090038999999, 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.695852469999998, 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.695657572000002, 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34552 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.552176086999999, 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013168394, 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34552 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167249+0000", 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167254+0000", 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2320000000000003e-06 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167273+0000", 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9502999999999999e-05 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167323+0000", 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9527999999999997e-05 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167346+0000", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3618999999999999e-05 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167513+0000", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166992 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.167758+0000", 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000244622 2024-09-21T22:51:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.169268+0000", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015101559999999999 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180086+0000", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010817937999999999 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180137+0000", 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0633999999999997e-05 2024-09-21T22:51:00.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180295+0000", 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015844899999999999 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.180418+0000", 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012272100000000001 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:34557 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.547233325000001, 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010897125000000001, 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34557 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172192+0000", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172195+0000", 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7099999999999999e-06 2024-09-21T22:51:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172210+0000", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4823e-05 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172240+0000", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0361999999999998e-05 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172255+0000", 2024-09-21T22:51:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552000000000001e-05 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172390+0000", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013588399999999999 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.172680+0000", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028962999999999998 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.174751+0000", 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020706689999999998 2024-09-21T22:51:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182868+0000", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081170249999999999 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.182936+0000", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8422000000000003e-05 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183021+0000", 2024-09-21T22:51:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5043000000000006e-05 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:45.183089+0000", 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8004999999999997e-05 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40788 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 190464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.54644001, 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012747681, 2024-09-21T22:51:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40788 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172985+0000", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172986+0000", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5499999999999998e-07 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172990+0000", 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5820000000000002e-06 2024-09-21T22:51:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.172998+0000", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7679999999999997e-06 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173022+0000", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3747000000000002e-05 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182235+0000", 2024-09-21T22:51:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092129110000000007 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182364+0000", 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128589 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185733+0000", 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003369529 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40789 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546393231, 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010153383, 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40789 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173032+0000", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173033+0000", 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6400000000000003e-07 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173035+0000", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8389999999999998e-06 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173044+0000", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173049+0000", 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5330000000000002e-06 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.182955+0000", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099058489999999996 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183053+0000", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8072000000000006e-05 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183186+0000", 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013232700000000001 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40790 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546367755, 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010372809, 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40790 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173058+0000", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7399999999999998e-07 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173061+0000", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.853e-06 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173069+0000", 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3340000000000002e-06 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173077+0000", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9780000000000005e-06 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183219+0000", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010142004 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183319+0000", 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9908000000000006e-05 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183430+0000", 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111258 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40791 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546337591, 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010578489, 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40791 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:51:00.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173088+0000", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6899999999999998e-07 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173110+0000", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1763000000000002e-05 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173120+0000", 2024-09-21T22:51:00.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8740000000000007e-06 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173135+0000", 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5356999999999999e-05 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183465+0000", 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010329823 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183558+0000", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2762000000000004e-05 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183666+0000", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108441 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40792 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:51:00.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546276012, 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01076204, 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40792 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:51:00.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173149+0000", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173150+0000", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173163+0000", 2024-09-21T22:51:00.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3438000000000001e-05 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173173+0000", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5689999999999994e-06 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173180+0000", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7889999999999997e-06 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183703+0000", 2024-09-21T22:51:00.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010523341 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183800+0000", 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6836999999999995e-05 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183911+0000", 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011145300000000001 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40793 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 223232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546235244, 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010959662, 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40793 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173190+0000", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173191+0000", 2024-09-21T22:51:00.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1399999999999997e-07 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173193+0000", 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5830000000000001e-06 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173202+0000", 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4370000000000007e-06 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173209+0000", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7200000000000006e-06 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.183946+0000", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010736193999999999 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184038+0000", 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2070000000000004e-05 2024-09-21T22:51:00.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184150+0000", 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112144 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40794 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546206356999999, 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011173367, 2024-09-21T22:51:00.780 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40794 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173219+0000", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.89e-07 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173222+0000", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-06 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173230+0000", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173237+0000", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4200000000000001e-06 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184188+0000", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010950295000000001 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184281+0000", 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3764999999999994e-05 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184392+0000", 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011095999999999999 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40795 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546160543999999, 2024-09-21T22:51:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011369611, 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40795 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173265+0000", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8999999999999997e-07 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173277+0000", 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1144999999999999e-05 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173285+0000", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5399999999999996e-06 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173308+0000", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3237000000000001e-05 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184427+0000", 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011118303 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184521+0000", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4660999999999994e-05 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184634+0000", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113235 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40796 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:51:00.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546096433000001, 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011518128000000001, 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40796 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:51:00.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173329+0000", 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173330+0000", 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7700000000000004e-07 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173334+0000", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1200000000000004e-06 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173343+0000", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6369999999999994e-06 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173351+0000", 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.221e-06 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184666+0000", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011315733999999999 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184758+0000", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1173000000000003e-05 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184847+0000", 2024-09-21T22:51:00.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9566000000000003e-05 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40797 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546065200999999, 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011696465, 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40797 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173360+0000", 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173361+0000", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1200000000000003e-07 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173364+0000", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173372+0000", 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2369999999999992e-06 2024-09-21T22:51:00.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173378+0000", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8860000000000001e-06 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184880+0000", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011502603 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184971+0000", 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0486000000000001e-05 2024-09-21T22:51:00.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185057+0000", 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5853999999999995e-05 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40798 3.10 3:09570432:::rbd_data.111e4aac08c7.0000000000000015:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.546038833000001, 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011876305, 2024-09-21T22:51:00.797 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40798 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173387+0000", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7800000000000002e-07 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173389+0000", 2024-09-21T22:51:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3920000000000001e-06 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173397+0000", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.841e-06 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.173403+0000", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185089+0000", 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011685600000000001 2024-09-21T22:51:00.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185179+0000", 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0142999999999994e-05 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185263+0000", 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4204999999999998e-05 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40997 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 247808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.544747879999999, 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010222111000000001, 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40997 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174678+0000", 2024-09-21T22:51:00.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1399999999999998e-07 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174680+0000", 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8369999999999999e-06 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174684+0000", 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6970000000000001e-06 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174689+0000", 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3039999999999998e-06 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184727+0000", 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010038038000000001 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184816+0000", 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9507999999999998e-05 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.184900+0000", 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3412999999999996e-05 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:40998 3.a 3:51459e17:::rbd_data.111e4aac08c7.000000000000001a:10 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.544732360999999, 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01102735, 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 40998 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174693+0000", 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7000000000000001e-07 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174700+0000", 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3999999999999997e-06 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174705+0000", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0810000000000001e-06 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.174714+0000", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1169999999999993e-06 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185509+0000", 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010795157 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185617+0000", 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010754100000000001 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.185720+0000", 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010378399999999999 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41173 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.541579575, 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010408699, 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41173 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177846+0000", 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1900000000000003e-07 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177849+0000", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.965e-06 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177858+0000", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0229999999999998e-06 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177875+0000", 2024-09-21T22:51:00.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6489000000000001e-05 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187860+0000", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099856619999999993 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.187967+0000", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106217 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188255+0000", 2024-09-21T22:51:00.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000287824 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:41174 3.10 3:096e58bb:::rbd_data.111e4aac08c7.0000000000000024:10 [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.541539329000001, 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010877736000000001, 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:00.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 41174 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:00.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177886+0000", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177887+0000", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999995e-07 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177890+0000", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.507e-06 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177900+0000", 2024-09-21T22:51:00.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8130000000000007e-06 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.177907+0000", 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800000000000003e-06 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188332+0000", 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010425419999999999 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188450+0000", 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117992 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:47.188764+0000", 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031363700000000002 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:00.819 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:00.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:00.821 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:00.820+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:00.856 INFO:teuthology.orchestra.run.smithi094.stdout:3275 writesame 0x833aca8 thru 0x833f37a data_size 0x1(0x46d3 bytes) 2024-09-21T22:51:00.856 INFO:teuthology.orchestra.run.smithi094.stdout:3276 read 0x39c71a thru 0x3a590c (0x91f3 bytes) 2024-09-21T22:51:00.859 INFO:teuthology.orchestra.run.smithi094.stdout:3277 punch from 0x9debb4 to 0x9eaf3e, (0xc38a bytes) 2024-09-21T22:51:00.859 INFO:teuthology.orchestra.run.smithi094.stdout:3278 punch from 0x828d7f3 to 0x8292dce, (0x55db bytes) 2024-09-21T22:51:00.859 INFO:teuthology.orchestra.run.smithi094.stdout:3279 read 0x4e25f1f thru 0x4e29e12 (0x3ef4 bytes) 2024-09-21T22:51:00.859 INFO:teuthology.orchestra.run.smithi094.stdout:3280 read 0x31998f4 thru 0x31a8931 (0xf03e bytes) 2024-09-21T22:51:00.860 INFO:teuthology.orchestra.run.smithi094.stdout:3281 read 0xa45806 thru 0xa4693c (0x1137 bytes) 2024-09-21T22:51:00.860 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:00.868 INFO:teuthology.orchestra.run.smithi094.stdout:3283 write 0x881ecee thru 0x882c530 (0xd843 bytes) 2024-09-21T22:51:00.885 INFO:teuthology.orchestra.run.smithi094.stdout:3284 write 0x8e475c2 thru 0x8e4f69a (0x80d9 bytes) 2024-09-21T22:51:00.893 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:00.893 INFO:teuthology.orchestra.run.smithi094.stdout:3286 punch from 0x2f1c854 to 0x2f2062a, (0x3dd6 bytes) 2024-09-21T22:51:00.905 INFO:teuthology.orchestra.run.smithi094.stdout:3287 trunc from 0x8e4f69b to 0x843a38 2024-09-21T22:51:00.922 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:00.920+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:00.960 INFO:teuthology.orchestra.run.smithi094.stdout:3288 read 0x1aed94 thru 0x1b2830 (0x3a9d bytes) 2024-09-21T22:51:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:3289 punch from 0x493299 to 0x49775e, (0x44c5 bytes) 2024-09-21T22:51:00.963 INFO:teuthology.orchestra.run.smithi094.stdout:3290 punch from 0x7d2fec to 0x7d571c, (0x2730 bytes) 2024-09-21T22:51:00.970 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:00.970 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:00.970 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:00.970 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:00.970 INFO:teuthology.orchestra.run.smithi094.stdout:3291 trunc from 0x843a38 to 0x4d16171 2024-09-21T22:51:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:3293 punch from 0x937427 to 0x93a0ff, (0x2cd8 bytes) 2024-09-21T22:51:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:3294 punch from 0x17548a5 to 0x175ec7f, (0xa3da bytes) 2024-09-21T22:51:00.978 INFO:teuthology.orchestra.run.smithi094.stdout:3295 trunc from 0x4d16171 to 0x5b9733e 2024-09-21T22:51:00.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:00.996 INFO:teuthology.orchestra.run.smithi094.stdout:3296 trunc from 0x5b9733e to 0xb700cfb 2024-09-21T22:51:01.011 INFO:teuthology.orchestra.run.smithi094.stdout:3299 trunc from 0xb700cfb to 0x72798c9 2024-09-21T22:51:01.023 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:01.020+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:01.033 INFO:teuthology.orchestra.run.smithi094.stdout:3300 writesame 0xb1e3b55 thru 0xb1ee9d7 data_size 0x5(0xae83 bytes) 2024-09-21T22:51:01.033 INFO:teuthology.orchestra.run.smithi094.stdout:3301 write 0x90ffee thru 0x91cc8b (0xcc9e bytes) 2024-09-21T22:51:01.038 INFO:teuthology.orchestra.run.smithi094.stdout:3302 write 0x2d42154 thru 0x2d4819b (0x6048 bytes) 2024-09-21T22:51:01.044 INFO:teuthology.orchestra.run.smithi094.stdout:3303 punch from 0x2d05d7d to 0x2d15a6b, (0xfcee bytes) 2024-09-21T22:51:01.044 INFO:teuthology.orchestra.run.smithi094.stdout:3304 writesame 0x8121731 thru 0x81259be data_size 0x7(0x428e bytes) 2024-09-21T22:51:01.049 INFO:teuthology.orchestra.run.smithi094.stdout:3305 read 0x84bf458 thru 0x84c83d2 (0x8f7b bytes) 2024-09-21T22:51:01.060 INFO:teuthology.orchestra.run.smithi094.stdout:3306 writesame 0xd26e538 thru 0xd274138 data_size 0x3(0x5c01 bytes) 2024-09-21T22:51:01.060 INFO:teuthology.orchestra.run.smithi094.stdout:3307 writesame 0xbea6435 thru 0xbeb1f29 data_size 0x13(0xbaf5 bytes) 2024-09-21T22:51:01.066 INFO:teuthology.orchestra.run.smithi094.stdout:3308 write 0xaffa857 thru 0xb007318 (0xcac2 bytes) 2024-09-21T22:51:01.074 INFO:teuthology.orchestra.run.smithi094.stdout:3309 read 0x9d6f095 thru 0x9d7edb9 (0xfd25 bytes) 2024-09-21T22:51:01.074 INFO:teuthology.orchestra.run.smithi094.stdout:3310 write 0xc642afc thru 0xc644b0d (0x2012 bytes) 2024-09-21T22:51:01.077 INFO:teuthology.orchestra.run.smithi094.stdout:3312 writesame 0xc406e88 thru 0xc4129b2 data_size 0x25(0xbb2b bytes) 2024-09-21T22:51:01.085 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:01.087 INFO:teuthology.orchestra.run.smithi094.stdout:3314 trunc from 0xd274139 to 0xbf865b7 2024-09-21T22:51:01.104 INFO:teuthology.orchestra.run.smithi094.stdout:3315 write 0xca6aeb thru 0xcb1590 (0xaaa6 bytes) 2024-09-21T22:51:01.112 INFO:teuthology.orchestra.run.smithi094.stdout:3316 punch from 0x7bc4a03 to 0x7bc5394, (0x991 bytes) 2024-09-21T22:51:01.112 INFO:teuthology.orchestra.run.smithi094.stdout:3317 write 0x8577d7 thru 0x8630f3 (0xb91d bytes) 2024-09-21T22:51:01.119 INFO:teuthology.orchestra.run.smithi094.stdout:3319 punch from 0x9664726 to 0x9673f1a, (0xf7f4 bytes) 2024-09-21T22:51:01.119 INFO:teuthology.orchestra.run.smithi094.stdout:3321 write 0xad0e178 thru 0xad17078 (0x8f01 bytes) 2024-09-21T22:51:01.123 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:01.116+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:01.126 INFO:teuthology.orchestra.run.smithi094.stdout:3323 punch from 0x347f984 to 0x34842e1, (0x495d bytes) 2024-09-21T22:51:01.126 INFO:teuthology.orchestra.run.smithi094.stdout:3324 write 0x1c57925 thru 0x1c64830 (0xcf0c bytes) 2024-09-21T22:51:01.134 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:01.134 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:01.134 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:01.134 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:01.134 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:01.138 INFO:teuthology.orchestra.run.smithi094.stdout:3325 write 0x5494318 thru 0x54977d9 (0x34c2 bytes) 2024-09-21T22:51:01.143 INFO:teuthology.orchestra.run.smithi094.stdout:3326 read 0x4ae924e thru 0x4ae9699 (0x44c bytes) 2024-09-21T22:51:01.143 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:01.143 INFO:teuthology.orchestra.run.smithi094.stdout:3328 read 0xaabaf62 thru 0xaabbdea (0xe89 bytes) 2024-09-21T22:51:01.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:01.153 INFO:teuthology.orchestra.run.smithi094.stdout:3329 write 0xd4f3cb0 thru 0xd4fe9da (0xad2b bytes) 2024-09-21T22:51:01.177 INFO:teuthology.orchestra.run.smithi094.stdout:3330 trunc from 0xd4fe9db to 0x1121625 2024-09-21T22:51:01.205 INFO:teuthology.orchestra.run.smithi094.stdout:3331 read 0x5b074c thru 0x5b5c0a (0x54bf bytes) 2024-09-21T22:51:01.207 INFO:teuthology.orchestra.run.smithi094.stdout:3333 trunc from 0x1121625 to 0x15a8ca0 2024-09-21T22:51:01.224 INFO:teuthology.orchestra.run.smithi094.stdout:3334 write 0x2ac83b3 thru 0x2ad25e5 (0xa233 bytes) 2024-09-21T22:51:01.224 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:01.220+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:01.231 INFO:teuthology.orchestra.run.smithi094.stdout:3336 read 0x1123f79 thru 0x1132805 (0xe88d bytes) 2024-09-21T22:51:01.232 INFO:teuthology.orchestra.run.smithi094.stdout:3337 punch from 0x642d1f to 0x6497b9, (0x6a9a bytes) 2024-09-21T22:51:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:3341 punch from 0x1e65e79 to 0x1e67a9d, (0x1c24 bytes) 2024-09-21T22:51:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:3343 punch from 0x2363206 to 0x23729de, (0xf7d8 bytes) 2024-09-21T22:51:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:3344 read 0x292090c thru 0x2926985 (0x607a bytes) 2024-09-21T22:51:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:3346 clone 20 order 24 su 65536 sc 8 2024-09-21T22:51:01.284 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.80209657, 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:01.289 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.80205694, 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.801291454999999, 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.294 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:51:01.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:51:01.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:51:01.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.801077507, 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.800987513999999, 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:01.306 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.800926298, 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:51:01.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:51:01.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.240084928, 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:51:01.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.240071381, 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:51:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:51:01.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.240050998999999, 2024-09-21T22:51:01.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:51:01.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.24003757, 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:51:01.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.240023107000001, 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.240010482000001, 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.336 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239972082, 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:51:01.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.23994761, 2024-09-21T22:51:01.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:51:01.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:01.345 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239928745, 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:01.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:51:01.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239915507999999, 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:51:01.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:51:01.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239902535000001, 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:51:01.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:51:01.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239889741000001, 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:51:01.358 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.362 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.23987724, 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.363 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:51:01.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.239566814, 2024-09-21T22:51:01.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:51:01.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:01.373 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:01.373 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:01.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:01.375 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:01.368+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:01.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:01.475 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:01.468+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:01.550 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:01.550 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:01.551 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:01.551 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:01.562 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:01.562 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:01.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:01.578 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:01.579 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:01.572+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:01.596 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:01.596 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:01.612 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:01.612 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:01.633 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:01.633 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:01.654 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:01.654 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:01.675 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:01.675 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:01.676 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:01.672+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:01.692 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:01.692 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:01.695 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:01.695 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:01.708 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:01.777 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:01.776+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:01.872 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:01.872 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:01.878 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:01.872+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:01.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:01.979 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:01.972+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:02.079 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:02.076+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:02.179 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:02.172+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:02.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:51:02.239 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.239 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.255 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:02.286 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone19 from 0x2ad25e6 (overlap 0x843a38) to 0x0 2024-09-21T22:51:02.286 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:02.280+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:02.381 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:02.380+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:02.425 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.425 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:02.483 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:02.480+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:02.584 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:02.580+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:02.600 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.600 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:02.685 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:02.680+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:02.816 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:02.812+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:02.838 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T22:51:02.886 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:02.884+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:02.928 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:02.928 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:02.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:02.988 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:02.984+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:03.067 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.067 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.078 INFO:teuthology.orchestra.run.smithi094.stdout:3347 write 0xa0177af thru 0xa01972f (0x1f81 bytes) 2024-09-21T22:51:03.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:03.083 INFO:teuthology.orchestra.run.smithi094.stdout:3348 read 0x2adf7f6 thru 0x2aea788 (0xaf93 bytes) 2024-09-21T22:51:03.083 INFO:teuthology.orchestra.run.smithi094.stdout:3349 writesame 0x9b12cea thru 0x9b1a89f data_size 0x5(0x7bb6 bytes) 2024-09-21T22:51:03.100 INFO:teuthology.orchestra.run.smithi094.stdout:3350 trunc from 0xa019730 to 0xdc19acf 2024-09-21T22:51:03.111 INFO:teuthology.orchestra.run.smithi094.stdout:3351 punch from 0x22d10bb to 0x22d2967, (0x18ac bytes) 2024-09-21T22:51:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:3352 punch from 0x67a96e7 to 0x67b2280, (0x8b99 bytes) 2024-09-21T22:51:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:3353 read 0x859b2b8 thru 0x859d8a9 (0x25f2 bytes) 2024-09-21T22:51:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:3354 write 0x97cd7b8 thru 0x97d63be (0x8c07 bytes) 2024-09-21T22:51:03.121 INFO:teuthology.orchestra.run.smithi094.stdout:3355 writesame 0xbca975 thru 0xbd29a7 data_size 0x1(0x8033 bytes) 2024-09-21T22:51:03.129 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:03.128+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:03.176 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:51:03.181 INFO:teuthology.orchestra.run.smithi094.stdout:3356 write 0x222692 thru 0x225c6b (0x35da bytes) 2024-09-21T22:51:03.189 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:03.184+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:03.204 INFO:teuthology.orchestra.run.smithi094.stdout:3357 punch from 0x4c64518 to 0x4c6c084, (0x7b6c bytes) 2024-09-21T22:51:03.209 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.210 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.212 INFO:teuthology.orchestra.run.smithi094.stdout:3359 write 0xde46e05 thru 0xde4dd0d (0x6f09 bytes) 2024-09-21T22:51:03.214 INFO:teuthology.orchestra.run.smithi094.stdout:3360 writesame 0x9556072 thru 0x9565516 data_size 0x17(0xf4a5 bytes) 2024-09-21T22:51:03.221 INFO:teuthology.orchestra.run.smithi094.stdout:3361 punch from 0x41257fc to 0x412d506, (0x7d0a bytes) 2024-09-21T22:51:03.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:3362 trunc from 0xde4dd0e to 0x9f9dc60 2024-09-21T22:51:03.240 INFO:teuthology.orchestra.run.smithi094.stdout:3364 writesame 0x8b23875 thru 0x8b30986 data_size 0x7(0xd112 bytes) 2024-09-21T22:51:03.250 INFO:teuthology.orchestra.run.smithi094.stdout:3365 trunc from 0x9f9dc60 to 0x9a7a3bf 2024-09-21T22:51:03.277 INFO:teuthology.orchestra.run.smithi094.stdout:3367 writesame 0xc647577 thru 0xc64b200 data_size 0x52(0x3c8a bytes) 2024-09-21T22:51:03.278 INFO:teuthology.orchestra.run.smithi094.stdout:3368 write 0xb03095e thru 0xb03415e (0x3801 bytes) 2024-09-21T22:51:03.281 INFO:teuthology.orchestra.run.smithi094.stdout:3369 read 0x2f6925b thru 0x2f72603 (0x93a9 bytes) 2024-09-21T22:51:03.281 INFO:teuthology.orchestra.run.smithi094.stdout:3370 write 0x8ad3b4c thru 0x8add1e2 (0x9697 bytes) 2024-09-21T22:51:03.289 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:03.284+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:03.299 INFO:teuthology.orchestra.run.smithi094.stdout:3372 trunc from 0xc64b201 to 0x56387b2 2024-09-21T22:51:03.373 INFO:teuthology.orchestra.run.smithi094.stdout:3373 writesame 0xac480e6 thru 0xac5649a data_size 0x1b(0xe3b5 bytes) 2024-09-21T22:51:03.373 INFO:teuthology.orchestra.run.smithi094.stdout:3375 write 0x420ae9 thru 0x426849 (0x5d61 bytes) 2024-09-21T22:51:03.376 INFO:teuthology.orchestra.run.smithi094.stdout:3376 write 0x5bce388 thru 0x5bd27ad (0x4426 bytes) 2024-09-21T22:51:03.379 INFO:teuthology.orchestra.run.smithi094.stdout:3379 writesame 0x186afe0 thru 0x1876e82 data_size 0x1(0xbea3 bytes) 2024-09-21T22:51:03.390 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:03.388+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:03.402 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.402 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.408 INFO:teuthology.orchestra.run.smithi094.stdout:3380 write 0x931088f thru 0x9311ea2 (0x1614 bytes) 2024-09-21T22:51:03.415 INFO:teuthology.orchestra.run.smithi094.stdout:3381 punch from 0x96f530d to 0x96f6e23, (0x1b16 bytes) 2024-09-21T22:51:03.415 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:03.415 INFO:teuthology.orchestra.run.smithi094.stdout:3383 write 0x7d3d081 thru 0x7d479dc (0xa95c bytes) 2024-09-21T22:51:03.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:03.424 INFO:teuthology.orchestra.run.smithi094.stdout:3384 trunc from 0xac5649b to 0xd2603a9 2024-09-21T22:51:03.469 INFO:teuthology.orchestra.run.smithi094.stdout:3386 trunc from 0xd2603a9 to 0x1186b29 2024-09-21T22:51:03.498 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:03.501 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:03.500+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:03.521 INFO:teuthology.orchestra.run.smithi094.stdout:3388 write 0x5bc6578 thru 0x5bd1ddc (0xb865 bytes) 2024-09-21T22:51:03.525 INFO:teuthology.orchestra.run.smithi094.stdout:3389 read 0x43b906b thru 0x43c0836 (0x77cc bytes) 2024-09-21T22:51:03.560 INFO:teuthology.orchestra.run.smithi094.stdout:3391 write 0xec78e45 thru 0xec7eb49 (0x5d05 bytes) 2024-09-21T22:51:03.567 INFO:teuthology.orchestra.run.smithi094.stdout:3392 read 0x711457 thru 0x71f188 (0xdd32 bytes) 2024-09-21T22:51:03.568 INFO:teuthology.orchestra.run.smithi094.stdout:3393 write 0xa624604 thru 0xa631b89 (0xd586 bytes) 2024-09-21T22:51:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:3394 writesame 0xb3ccc33 thru 0xb3dc185 data_size 0x1(0xf553 bytes) 2024-09-21T22:51:03.585 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.585 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.591 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:03.584+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:03.600 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:03.670 INFO:teuthology.orchestra.run.smithi094.stdout:3395 trunc from 0xec7eb4a to 0x2a05807 2024-09-21T22:51:03.692 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:03.688+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:03.700 INFO:teuthology.orchestra.run.smithi094.stdout:3396 read 0x77abe thru 0x7816b (0x6ae bytes) 2024-09-21T22:51:03.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:03.733 INFO:teuthology.orchestra.run.smithi094.stdout:3399 writesame 0xabad0ae thru 0xabb75c6 data_size 0x5(0xa519 bytes) 2024-09-21T22:51:03.753 INFO:teuthology.orchestra.run.smithi094.stdout:3400 write 0xe620d23 thru 0xe62dbb5 (0xce93 bytes) 2024-09-21T22:51:03.759 INFO:teuthology.orchestra.run.smithi094.stdout:3401 write 0x5fcce2b thru 0x5fd5128 (0x82fe bytes) 2024-09-21T22:51:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:3402 writesame 0x72e2a4a thru 0x72f24a1 data_size 0x4(0xfa58 bytes) 2024-09-21T22:51:03.768 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.768 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.770 INFO:teuthology.orchestra.run.smithi094.stdout:3404 punch from 0xcfa3036 to 0xcfb13ec, (0xe3b6 bytes) 2024-09-21T22:51:03.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:03.792 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:03.788+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:03.794 INFO:teuthology.orchestra.run.smithi094.stdout:3405 trunc from 0xe62dbb6 to 0x471a445 2024-09-21T22:51:03.843 INFO:teuthology.orchestra.run.smithi094.stdout:3406 write 0xea205d9 thru 0xea2da35 (0xd45d bytes) 2024-09-21T22:51:03.848 INFO:teuthology.orchestra.run.smithi094.stdout:3407 read 0x313a30 thru 0x319ae0 (0x60b1 bytes) 2024-09-21T22:51:03.850 INFO:teuthology.orchestra.run.smithi094.stdout:3410 punch from 0x84d5290 to 0x84d9d25, (0x4a95 bytes) 2024-09-21T22:51:03.850 INFO:teuthology.orchestra.run.smithi094.stdout:3411 writesame 0xdbb7c8 thru 0xdc27ee data_size 0x1(0x7027 bytes) 2024-09-21T22:51:03.872 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:03.872 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:03.872 INFO:teuthology.orchestra.run.smithi094.stdout:3414 read 0xa29e53f thru 0xa29f329 (0xdeb bytes) 2024-09-21T22:51:03.872 INFO:teuthology.orchestra.run.smithi094.stdout:3415 write 0x26488e4 thru 0x2651ff3 (0x9710 bytes) 2024-09-21T22:51:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:3416 clone 21 order 18 su 4096 sc 12 2024-09-21T22:51:03.880 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:03.893 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:03.888+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:03.896 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:03.896 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:03.913 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:03.913 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:03.929 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:03.929 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:03.948 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:03.948 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:03.949 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:03.962 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:03.969 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:03.990 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:03.990 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:04.006 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:04.004+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:04.094 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:04.092+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:04.128 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:04.128 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:04.128 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:04.128 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:04.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:04.169 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:51:04.169 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:51:04.170 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-21T22:51:04.170 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T22:51:04.170 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T22:51:04.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-21T22:51:04.229 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:04.228+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:04.296 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:04.292+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:04.306 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:04.306 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:04.306 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:04.306 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:04.306 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:04.308 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:04.308 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T22:51:04.308 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:04.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:04.379 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone20 from 0xea2da36 (overlap 0x1186b29) to 0x0 2024-09-21T22:51:04.417 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:04.416+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:04.497 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:04.492+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:04.499 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 71.560362628000007, 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015355153000002, 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:51:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015290769, 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015227319000001, 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:04.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015164141, 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:04.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015107800999999, 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:04.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:04.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015066058999999, 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:04.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.015005458000001, 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014925022, 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014848912000001, 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014761838999998, 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014718350999999, 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014684713000001, 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014650947, 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014619663000001, 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014574308, 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014542811999998, 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.014508030999998, 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.9965625, 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.996501456000001, 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:04.595 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:04.596 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T22:51:04.596 INFO:teuthology.orchestra.run.smithi094.stdout:3417 punch from 0x334bdf5 to 0x335bb1c, (0xfd27 bytes) 2024-09-21T22:51:04.596 INFO:teuthology.orchestra.run.smithi094.stdout:3418 write 0xc6a4b6d thru 0xc6b39e7 (0xee7b bytes) 2024-09-21T22:51:04.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:04.617 INFO:teuthology.orchestra.run.smithi094.stdout:3419 flatten 2024-09-21T22:51:04.642 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:04.640+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:04.699 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:04.696+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:04.800 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:04.792+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:04.809 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:04.810 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:04.810 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:04.810 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:04.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:04.913 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:04.912+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:05.001 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:04.996+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:05.042 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:05.042 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:05.061 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:05.105 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:05.104+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:05.203 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:05.200+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:05.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.246855322999998, 2024-09-21T22:51:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:05.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:05.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.242617754000001, 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.274 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:05.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:05.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:05.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:05.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.242422856000001, 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:05.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:05.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:05.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:05.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:05.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68484 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 356352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62787611899999995, 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033448665000000002, 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68484 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638316+0000", 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0100000000000001e-06 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638322+0000", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169999999999997e-06 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638342+0000", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.995e-05 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638370+0000", 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7985000000000001e-05 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671391+0000", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033021259999999997 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671483+0000", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1380999999999998e-05 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671763+0000", 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028066199999999999 2024-09-21T22:51:05.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68541 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62780041099999995, 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033895258999999997, 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68541 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638391+0000", 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638398+0000", 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9770000000000003e-06 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638421+0000", 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2578999999999998e-05 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638436+0000", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4620000000000001e-05 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671874+0000", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033438273999999997 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671995+0000", 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120567 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672286+0000", 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029103199999999999 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68542 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 49152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62773581300000003, 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034317184000000001, 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68542 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638456+0000", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.567e-06 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638462+0000", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9830000000000001e-06 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638487+0000", 2024-09-21T22:51:05.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4199999999999999e-05 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638501+0000", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4489e-05 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672333+0000", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033832167000000003 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672439+0000", 2024-09-21T22:51:05.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105635 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672772+0000", 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033314299999999999 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68543 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62767052400000001, 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034634933999999999, 2024-09-21T22:51:05.300 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68543 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638521+0000", 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1060000000000001e-06 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638527+0000", 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638546+0000", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8635999999999999e-05 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638559+0000", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2733e-05 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672814+0000", 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034255173 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672909+0000", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4581999999999997e-05 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673155+0000", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024655400000000001 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68544 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 163840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62761318899999996, 2024-09-21T22:51:05.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034985167999999997, 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68544 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638579+0000", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006e-06 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638584+0000", 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6840000000000001e-06 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638604+0000", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9859000000000001e-05 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638618+0000", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.428e-05 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673193+0000", 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034575074999999997 2024-09-21T22:51:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673252+0000", 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8288000000000002e-05 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673563+0000", 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031097600000000003 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68545 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 196608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62747432000000003, 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035408569000000001, 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68545 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638718+0000", 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638725+0000", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8299999999999998e-06 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638746+0000", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1566e-05 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638760+0000", 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673630+0000", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034869372000000003 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673803+0000", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000173435 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674125+0000", 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032173299999999999 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68546 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 245760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62739823500000003, 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035763809000000001, 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68546 2024-09-21T22:51:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638794+0000", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4529999999999999e-06 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638815+0000", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1418999999999999e-05 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638837+0000", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1855999999999999e-05 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638851+0000", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.341e-05 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674168+0000", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035317861999999998 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674234+0000", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5946999999999997e-05 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674556+0000", 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032186200000000002 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68547 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 278528~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62731769800000003, 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03614469, 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68547 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638874+0000", 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638881+0000", 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9999999999999999e-06 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638901+0000", 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999999e-05 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638915+0000", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4532000000000001e-05 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674611+0000", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035696033000000002 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674712+0000", 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010040299999999999 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675018+0000", 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030589199999999999 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68548 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62725697300000005, 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036468652999999997, 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68548 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638935+0000", 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638941+0000", 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638959+0000", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8431999999999999e-05 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638973+0000", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3813e-05 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675083+0000", 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036110541000000003 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675199+0000", 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115926 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675402+0000", 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020310400000000001 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68605 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.627199795, 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036815352000000003, 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68605 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638992+0000", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638998+0000", 2024-09-21T22:51:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4370000000000004e-06 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639017+0000", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639055+0000", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7441999999999998e-05 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675453+0000", 2024-09-21T22:51:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036398463999999998 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675531+0000", 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7575000000000002e-05 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675806+0000", 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027533800000000003 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68606 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62711508999999999, 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036991366999999997, 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68606 2024-09-21T22:51:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639077+0000", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639085+0000", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7559999999999999e-06 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639118+0000", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3705999999999999e-05 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639148+0000", 2024-09-21T22:51:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0162999999999999e-05 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675849+0000", 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036700993000000001 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675907+0000", 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8032999999999998e-05 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676067+0000", 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015953300000000001 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68607 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 86016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62702107299999998, 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037187378, 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68607 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639171+0000", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639178+0000", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6699999999999997e-06 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639198+0000", 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0132e-05 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639212+0000", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4294000000000001e-05 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676124+0000", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036912568 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676193+0000", 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8821999999999999e-05 2024-09-21T22:51:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676357+0000", 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016368200000000001 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68608 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62695706200000001, 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037593025000000002, 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68608 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639235+0000", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.034e-06 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639241+0000", 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5830000000000003e-06 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639268+0000", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6741999999999999e-05 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639281+0000", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.344e-05 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676415+0000", 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037133806999999998 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676541+0000", 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012614500000000001 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676827+0000", 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028527400000000002 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68609 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62688984699999994, 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037858175000000001, 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68609 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639302+0000", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0669999999999999e-06 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639309+0000", 2024-09-21T22:51:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7429999999999996e-06 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639327+0000", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8573e-05 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639341+0000", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3286e-05 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676887+0000", 2024-09-21T22:51:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037546698000000003 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676966+0000", 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8611999999999999e-05 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677159+0000", 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000193196 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68610 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62683139799999998, 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038202400999999997, 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68610 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639360+0000", 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639367+0000", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1759999999999998e-06 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639385+0000", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8454e-05 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639399+0000", 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3835999999999999e-05 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677208+0000", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037809295999999999 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677317+0000", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108682 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677562+0000", 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024489100000000001 2024-09-21T22:51:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68611 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62677293000000001, 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038561235999999999, 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68611 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639419+0000", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639425+0000", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639443+0000", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8675999999999999e-05 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639457+0000", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4083000000000001e-05 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677605+0000", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038147542999999999 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677713+0000", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108529 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677979+0000", 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026561500000000002 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68612 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.62668469999999998, 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038912731999999998, 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68612 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639508+0000", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639514+0000", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3500000000000002e-06 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639534+0000", 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0199000000000001e-05 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639549+0000", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4056000000000001e-05 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678034+0000", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038485761 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678166+0000", 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131719 2024-09-21T22:51:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678419+0000", 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025270300000000001 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:05.359 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:05.360 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:05.356+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:05.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:05.461 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:05.456+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:84437 3.6 3.92bdcc86 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:05.544896+0000", 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0015729209999999999, 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015957949999999999, 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 84437 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544896+0000", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544896+0000", 2024-09-21T22:51:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544896+0000", 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544897+0000", 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1400000000000001e-06 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544900+0000", 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1080000000000001e-06 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545115+0000", 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000214191 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:84438 3.6 3.92bdcc86 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:05.545127+0000", 2024-09-21T22:51:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0013417290000000001, 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001430321, 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 84438 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545127+0000", 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545127+0000", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545128+0000", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2900000000000002e-07 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545132+0000", 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9269999999999998e-06 2024-09-21T22:51:05.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545140+0000", 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7249999999999997e-06 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.545144+0000", 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1160000000000001e-06 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:84434 3.6 3:6133bd49:::rbd_data.111e28e769d9.0000000000000004:17 [read 8413184~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:05.544865+0000", 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016039089999999999, 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001755277, 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 84434 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544865+0000", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544865+0000", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544865+0000", 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1899999999999999e-07 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544866+0000", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544870+0000", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1520000000000001e-06 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544872+0000", 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6410000000000002e-06 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.546467+0000", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015953460000000001 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.546539+0000", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1470999999999994e-05 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.546625+0000", 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5636000000000005e-05 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:84435 3.6 3.92bdcc86 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:05.544875+0000", 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001593428, 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001795615, 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 84435 2024-09-21T22:51:05.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544875+0000", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544875+0000", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544876+0000", 2024-09-21T22:51:05.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.03e-07 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544877+0000", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4839999999999999e-06 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544880+0000", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1810000000000001e-06 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544883+0000", 2024-09-21T22:51:05.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5019999999999999e-06 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.546667+0000", 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017841809999999999 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:84436 3.6 3.92bdcc86 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:05.544886+0000", 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001582977, 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001822008, 2024-09-21T22:51:05.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 84436 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544886+0000", 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544886+0000", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544886+0000", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544887+0000", 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2950000000000001e-06 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544890+0000", 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:05.544893+0000", 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4339999999999999e-06 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 5 2024-09-21T22:51:05.564 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:05.565 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:05.564+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:05.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:05.666 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:05.660+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:05.738 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:05.738 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:05.738 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:05.738 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:05.738 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:05.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:05.796 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:05.792+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:05.868 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:05.864+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.483036112000001, 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.482996482000001, 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.482230996999998, 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.482017049, 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.481927056, 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.481865840000001, 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:51:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:51:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.921024469999999, 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:51:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.921010923000001, 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920990540999998, 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920977111999999, 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920962649, 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920950024, 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:51:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920911623999999, 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920887151999999, 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:51:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:51:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920868287000001, 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:51:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.92085505, 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:51:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:51:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920842077, 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:51:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:51:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920829283, 2024-09-21T22:51:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:51:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:51:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920816781999999, 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:51:06.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.920506356000001, 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:06.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:06.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:51:06.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:51:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:06.052 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:06.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:06.054 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:06.052+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:06.054 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:06.155 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:06.152+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:51:06.255 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:06.248+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:06.262 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:06.262 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:06.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:06.303 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:06.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:06.324 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:06.347 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:06.347 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:06.356 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:06.352+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:06.368 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:06.368 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:06.392 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:06.416 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:06.434 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:06.434 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:06.438 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:06.438 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:06.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:06.458 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:06.459 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:06.459 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:06.452+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:06.557 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:06.552+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:06.561 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:06.562 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:06.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:06.660 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:06.656+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:06.740 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:06.740 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:06.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:06.796 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:06.792+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:06.860 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:06.856+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:06.933 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:06.934 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:06.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:06.959 INFO:teuthology.orchestra.run.smithi094.stdout:3420 punch from 0xb6cb44b to 0xb6d571b, (0xa2d0 bytes) 2024-09-21T22:51:06.959 INFO:teuthology.orchestra.run.smithi094.stdout:3421 read 0x5d67b87 thru 0x5d73733 (0xbbad bytes) 2024-09-21T22:51:06.961 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:06.956+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:06.970 INFO:teuthology.orchestra.run.smithi094.stdout:3422 trunc from 0xea2da36 to 0x4c86080 2024-09-21T22:51:07.000 INFO:teuthology.orchestra.run.smithi094.stdout:3423 write 0x8946a5a thru 0x8949cdb (0x3282 bytes) 2024-09-21T22:51:07.010 INFO:teuthology.orchestra.run.smithi094.stdout:3424 read 0x37ea600 thru 0x37ef0ed (0x4aee bytes) 2024-09-21T22:51:07.010 INFO:teuthology.orchestra.run.smithi094.stdout:3427 punch from 0x888830e to 0x8888966, (0x658 bytes) 2024-09-21T22:51:07.010 INFO:teuthology.orchestra.run.smithi094.stdout:3429 read 0x37b8ae9 thru 0x37c4151 (0xb669 bytes) 2024-09-21T22:51:07.023 INFO:teuthology.orchestra.run.smithi094.stdout:3432 write 0xcf19c34 thru 0xcf256ea (0xbab7 bytes) 2024-09-21T22:51:07.041 INFO:teuthology.orchestra.run.smithi094.stdout:3434 writesame 0x4780fe7 thru 0x4790d3c data_size 0x1b(0xfd56 bytes) 2024-09-21T22:51:07.058 INFO:teuthology.orchestra.run.smithi094.stdout:3435 punch from 0xa0cfdfc to 0xa0de0e9, (0xe2ed bytes) 2024-09-21T22:51:07.058 INFO:teuthology.orchestra.run.smithi094.stdout:3437 punch from 0x2079323 to 0x2087618, (0xe2f5 bytes) 2024-09-21T22:51:07.058 INFO:teuthology.orchestra.run.smithi094.stdout:3438 write 0xd81f25 thru 0xd8f0db (0xd1b7 bytes) 2024-09-21T22:51:07.061 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:07.060+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:07.065 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:07.065 INFO:teuthology.orchestra.run.smithi094.stdout:3441 writesame 0x6d3191 thru 0x6d5fff data_size 0x1(0x2e6f bytes) 2024-09-21T22:51:07.075 INFO:teuthology.orchestra.run.smithi094.stdout:3442 trunc from 0xcf256eb to 0x750031f 2024-09-21T22:51:07.090 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.090 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.103 INFO:teuthology.orchestra.run.smithi094.stdout:3443 trunc from 0x750031f to 0x6af7f34 2024-09-21T22:51:07.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:3444 punch from 0x102508e to 0x1034bfd, (0xfb6f bytes) 2024-09-21T22:51:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:3445 read 0x51614a9 thru 0x5165d01 (0x4859 bytes) 2024-09-21T22:51:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:3446 writesame 0x58db806 thru 0x58e62bc data_size 0x1d(0xaab7 bytes) 2024-09-21T22:51:07.128 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:07.128 INFO:teuthology.orchestra.run.smithi094.stdout:3449 read 0x68779b4 thru 0x688102e (0x967b bytes) 2024-09-21T22:51:07.128 INFO:teuthology.orchestra.run.smithi094.stdout:3450 writesame 0x896caa thru 0x8a423e data_size 0x49(0xd595 bytes) 2024-09-21T22:51:07.137 INFO:teuthology.orchestra.run.smithi094.stdout:3452 punch from 0x157140c to 0x1577b56, (0x674a bytes) 2024-09-21T22:51:07.137 INFO:teuthology.orchestra.run.smithi094.stdout:3453 read 0x5fb2d5b thru 0x5fbadff (0x80a5 bytes) 2024-09-21T22:51:07.146 INFO:teuthology.orchestra.run.smithi094.stdout:3454 write 0x74ad267 thru 0x74b2c4f (0x59e9 bytes) 2024-09-21T22:51:07.158 INFO:teuthology.orchestra.run.smithi094.stdout:3455 read 0x507351e thru 0x507c3b0 (0x8e93 bytes) 2024-09-21T22:51:07.158 INFO:teuthology.orchestra.run.smithi094.stdout:3457 read 0x625d661 thru 0x6261916 (0x42b6 bytes) 2024-09-21T22:51:07.159 INFO:teuthology.orchestra.run.smithi094.stdout:3458 write 0x30c0727 thru 0x30c0c17 (0x4f1 bytes) 2024-09-21T22:51:07.162 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:07.156+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:07.163 INFO:teuthology.orchestra.run.smithi094.stdout:3459 read 0x39684d4 thru 0x3976a97 (0xe5c4 bytes) 2024-09-21T22:51:07.177 INFO:teuthology.orchestra.run.smithi094.stdout:3461 write 0xd151f64 thru 0xd154461 (0x24fe bytes) 2024-09-21T22:51:07.186 INFO:teuthology.orchestra.run.smithi094.stdout:3462 read 0x117eaa9 thru 0x1187b2f (0x9087 bytes) 2024-09-21T22:51:07.186 INFO:teuthology.orchestra.run.smithi094.stdout:3464 writesame 0x95c2ea8 thru 0x95d2bae data_size 0x5(0xfd07 bytes) 2024-09-21T22:51:07.208 INFO:teuthology.orchestra.run.smithi094.stdout:3465 write 0xa7cc264 thru 0xa7d9e4c (0xdbe9 bytes) 2024-09-21T22:51:07.225 INFO:teuthology.orchestra.run.smithi094.stdout:3467 write 0x2dd290 thru 0x2e24e4 (0x5255 bytes) 2024-09-21T22:51:07.232 INFO:teuthology.orchestra.run.smithi094.stdout:3469 punch from 0x332c416 to 0x3335680, (0x926a bytes) 2024-09-21T22:51:07.232 INFO:teuthology.orchestra.run.smithi094.stdout:3473 write 0x8edb85 thru 0x8f78e3 (0x9d5f bytes) 2024-09-21T22:51:07.244 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.244 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.255 INFO:teuthology.orchestra.run.smithi094.stdout:3474 trunc from 0xd154462 to 0x1722987 2024-09-21T22:51:07.262 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:07.256+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:07.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:07.315 INFO:teuthology.orchestra.run.smithi094.stdout:3475 trunc from 0x1722987 to 0x8770413 2024-09-21T22:51:07.322 INFO:teuthology.orchestra.run.smithi094.stdout:3476 read 0x37b5b90 thru 0x37c1d1f (0xc190 bytes) 2024-09-21T22:51:07.330 INFO:teuthology.orchestra.run.smithi094.stdout:3477 write 0x8eae4a3 thru 0x8eb16ee (0x324c bytes) 2024-09-21T22:51:07.354 INFO:teuthology.orchestra.run.smithi094.stdout:3479 writesame 0xee33ed6 thru 0xee4222a data_size 0x13(0xe355 bytes) 2024-09-21T22:51:07.363 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:07.360+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:07.370 INFO:teuthology.orchestra.run.smithi094.stdout:3480 trunc from 0xee4222b to 0x1b39bad 2024-09-21T22:51:07.403 INFO:teuthology.orchestra.run.smithi094.stdout:3481 write 0x87b312d thru 0x87c23a5 (0xf279 bytes) 2024-09-21T22:51:07.423 INFO:teuthology.orchestra.run.smithi094.stdout:3483 punch from 0x2fbe4f2 to 0x2fcc6dd, (0xe1eb bytes) 2024-09-21T22:51:07.423 INFO:teuthology.orchestra.run.smithi094.stdout:3484 write 0x1767a2 thru 0x17e557 (0x7db6 bytes) 2024-09-21T22:51:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.441 INFO:teuthology.orchestra.run.smithi094.stdout:3487 writesame 0xbc8389b thru 0xbc83e71 data_size 0xd(0x5d7 bytes) 2024-09-21T22:51:07.442 INFO:teuthology.orchestra.run.smithi094.stdout:3488 write 0x81736e4 thru 0x817eb4d (0xb46a bytes) 2024-09-21T22:51:07.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:07.474 INFO:teuthology.orchestra.run.smithi094.stdout:3489 trunc from 0xbc83e72 to 0x2483c4a 2024-09-21T22:51:07.474 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:07.468+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:07.520 INFO:teuthology.orchestra.run.smithi094.stdout:3491 write 0xbc5fa7f thru 0xbc662e7 (0x6869 bytes) 2024-09-21T22:51:07.544 INFO:teuthology.orchestra.run.smithi094.stdout:3493 write 0xd40c6ad thru 0xd413a37 (0x738b bytes) 2024-09-21T22:51:07.557 INFO:teuthology.orchestra.run.smithi094.stdout:3494 trunc from 0xd413a38 to 0xd655809 2024-09-21T22:51:07.565 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:07.560+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:07.574 INFO:teuthology.orchestra.run.smithi094.stdout:3495 write 0xea43dcf thru 0xea522e6 (0xe518 bytes) 2024-09-21T22:51:07.594 INFO:teuthology.orchestra.run.smithi094.stdout:3497 write 0x6b280df thru 0x6b32689 (0xa5ab bytes) 2024-09-21T22:51:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.612 INFO:teuthology.orchestra.run.smithi094.stdout:3498 writesame 0xeacbe3 thru 0xeacbfc data_size 0x1(0x1a bytes) 2024-09-21T22:51:07.614 INFO:teuthology.orchestra.run.smithi094.stdout:3499 punch from 0xe0579b6 to 0xe060edc, (0x9526 bytes) 2024-09-21T22:51:07.614 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:3501 trunc from 0xea522e7 to 0x59ea7eb 2024-09-21T22:51:07.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:07.657 INFO:teuthology.orchestra.run.smithi094.stdout:3502 trunc from 0x59ea7eb to 0x439b771 2024-09-21T22:51:07.664 INFO:teuthology.orchestra.run.smithi094.stdout:3504 write 0x38e5f9 thru 0x396425 (0x7e2d bytes) 2024-09-21T22:51:07.671 INFO:teuthology.orchestra.run.smithi094.stdout:3505 read 0x2b41adf thru 0x2b5047b (0xe99d bytes) 2024-09-21T22:51:07.671 INFO:teuthology.orchestra.run.smithi094.stdout:3507 trunc from 0x439b771 to 0x40c3006 2024-09-21T22:51:07.674 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:07.672+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:07.693 INFO:teuthology.orchestra.run.smithi094.stdout:3509 writesame 0xa617a6b thru 0xa6245b3 data_size 0x53(0xcb49 bytes) 2024-09-21T22:51:07.693 INFO:teuthology.orchestra.run.smithi094.stdout:3510 write 0x9d98a6b thru 0x9da84a2 (0xfa38 bytes) 2024-09-21T22:51:07.712 INFO:teuthology.orchestra.run.smithi094.stdout:3511 punch from 0x27f7332 to 0x27fddc7, (0x6a95 bytes) 2024-09-21T22:51:07.712 INFO:teuthology.orchestra.run.smithi094.stdout:3513 read 0x8c116af thru 0x8c20d13 (0xf665 bytes) 2024-09-21T22:51:07.712 INFO:teuthology.orchestra.run.smithi094.stdout:3514 write 0x4130e0a thru 0x4131363 (0x55a bytes) 2024-09-21T22:51:07.716 INFO:teuthology.orchestra.run.smithi094.stdout:3515 write 0x8122107 thru 0x81291cd (0x70c7 bytes) 2024-09-21T22:51:07.729 INFO:teuthology.orchestra.run.smithi094.stdout:3516 read 0x38aa173 thru 0x38b23bc (0x824a bytes) 2024-09-21T22:51:07.729 INFO:teuthology.orchestra.run.smithi094.stdout:3517 write 0x351f2af thru 0x35250f0 (0x5e42 bytes) 2024-09-21T22:51:07.739 INFO:teuthology.orchestra.run.smithi094.stdout:3518 write 0x4138cf6 thru 0x413c0f9 (0x3404 bytes) 2024-09-21T22:51:07.748 INFO:teuthology.orchestra.run.smithi094.stdout:3521 trunc from 0xa6245b4 to 0x9cad4ee 2024-09-21T22:51:07.765 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:07.764+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:07.769 INFO:teuthology.orchestra.run.smithi094.stdout:3523 trunc from 0x9cad4ee to 0xea6fa56 2024-09-21T22:51:07.772 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.772 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.779 INFO:teuthology.orchestra.run.smithi094.stdout:3524 punch from 0x5b7d01c to 0x5b88ef0, (0xbed4 bytes) 2024-09-21T22:51:07.779 INFO:teuthology.orchestra.run.smithi094.stdout:3525 read 0x6a70faa thru 0x6a7c0d3 (0xb12a bytes) 2024-09-21T22:51:07.779 INFO:teuthology.orchestra.run.smithi094.stdout:3526 read 0x6d285d0 thru 0x6d2c91e (0x434f bytes) 2024-09-21T22:51:07.779 INFO:teuthology.orchestra.run.smithi094.stdout:3527 punch from 0x27f9be7 to 0x27fd29f, (0x36b8 bytes) 2024-09-21T22:51:07.779 INFO:teuthology.orchestra.run.smithi094.stdout:3528 write 0xd0fd3e7 thru 0xd1011c2 (0x3ddc bytes) 2024-09-21T22:51:07.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3530 read 0x1c5793a thru 0x1c63ea7 (0xc56e bytes) 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3531 punch from 0xaae4449 to 0xaae720b, (0x2dc2 bytes) 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3532 read 0xda954b9 thru 0xda9b22e (0x5d76 bytes) 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3534 punch from 0x82c423 to 0x8357e4, (0x93c1 bytes) 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3535 read 0x497837d thru 0x4980498 (0x811c bytes) 2024-09-21T22:51:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:3536 write 0x9aa7b18 thru 0x9ab1149 (0x9632 bytes) 2024-09-21T22:51:07.804 INFO:teuthology.orchestra.run.smithi094.stdout:3539 punch from 0x5010cb4 to 0x501802d, (0x7379 bytes) 2024-09-21T22:51:07.804 INFO:teuthology.orchestra.run.smithi094.stdout:3540 writesame 0x3eb45ce thru 0x3ebc6bf data_size 0x5(0x80f2 bytes) 2024-09-21T22:51:07.831 INFO:teuthology.orchestra.run.smithi094.stdout:3541 trunc from 0xea6fa56 to 0x405cef3 2024-09-21T22:51:07.856 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:07.856 INFO:teuthology.orchestra.run.smithi094.stdout:3543 read 0x40ac4 thru 0x45aac (0x4fe9 bytes) 2024-09-21T22:51:07.864 INFO:teuthology.orchestra.run.smithi094.stdout:3544 write 0x496e96b thru 0x497c410 (0xdaa6 bytes) 2024-09-21T22:51:07.873 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:07.872+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:07.886 INFO:teuthology.orchestra.run.smithi094.stdout:3546 trunc from 0x497c411 to 0x5fb44b 2024-09-21T22:51:07.932 INFO:teuthology.orchestra.run.smithi094.stdout:3547 trunc from 0x5fb44b to 0x74923af 2024-09-21T22:51:07.955 INFO:teuthology.orchestra.run.smithi094.stdout:3549 write 0xc60b5ee thru 0xc60f3ae (0x3dc1 bytes) 2024-09-21T22:51:07.964 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:07.964 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:07.964 INFO:teuthology.orchestra.run.smithi094.stdout:3551 write 0xa2da28 thru 0xa3bb73 (0xe14c bytes) 2024-09-21T22:51:07.967 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:07.964+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:07.980 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:07.984 INFO:teuthology.orchestra.run.smithi094.stdout:3552 punch from 0x24e450e to 0x24ef817, (0xb309 bytes) 2024-09-21T22:51:07.984 INFO:teuthology.orchestra.run.smithi094.stdout:3553 writesame 0xa4f246a thru 0xa4f294f data_size 0x13(0x4e6 bytes) 2024-09-21T22:51:07.998 INFO:teuthology.orchestra.run.smithi094.stdout:3554 trunc from 0xc60f3af to 0x67a028c 2024-09-21T22:51:08.024 INFO:teuthology.orchestra.run.smithi094.stdout:3555 trunc from 0x67a028c to 0xbd11e5a 2024-09-21T22:51:08.032 INFO:teuthology.orchestra.run.smithi094.stdout:3557 read 0x4ad5d56 thru 0x4adea65 (0x8d10 bytes) 2024-09-21T22:51:08.032 INFO:teuthology.orchestra.run.smithi094.stdout:3559 write 0xb211d8e thru 0xb21ac5f (0x8ed2 bytes) 2024-09-21T22:51:08.048 INFO:teuthology.orchestra.run.smithi094.stdout:3560 read 0x2dbdf0b thru 0x2dbf84d (0x1943 bytes) 2024-09-21T22:51:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:3562 writesame 0xe24ad8a thru 0xe254111 data_size 0x4(0x9388 bytes) 2024-09-21T22:51:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:3563 writesame 0x6e2073f thru 0x6e2ee74 data_size 0xf(0xe736 bytes) 2024-09-21T22:51:08.068 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:08.060+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:3566 read 0x5780181 thru 0x57846ff (0x457f bytes) 2024-09-21T22:51:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:3568 trunc from 0xe254112 to 0x6ded60 2024-09-21T22:51:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:3569 writesame 0x5080859 thru 0x508aacf data_size 0x13(0xa277 bytes) 2024-09-21T22:51:08.155 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:08.155 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:3570 write 0x71684a9 thru 0x7172f89 (0xaae1 bytes) 2024-09-21T22:51:08.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:08.169 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:08.164+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:3571 trunc from 0x7172f8a to 0x6409fb6 2024-09-21T22:51:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:3572 read 0x4e24704 thru 0x4e34552 (0xfe4f bytes) 2024-09-21T22:51:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:3573 write 0x369a222 thru 0x369d8b1 (0x3690 bytes) 2024-09-21T22:51:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:3574 write 0x5cb93c9 thru 0x5cc88bd (0xf4f5 bytes) 2024-09-21T22:51:08.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:51:08.245 INFO:teuthology.orchestra.run.smithi094.stdout:3575 writesame 0x2969bed thru 0x296adaf data_size 0x1(0x11c3 bytes) 2024-09-21T22:51:08.253 INFO:teuthology.orchestra.run.smithi094.stdout:3576 punch from 0x1e13301 to 0x1e194fc, (0x61fb bytes) 2024-09-21T22:51:08.253 INFO:teuthology.orchestra.run.smithi094.stdout:3577 read 0x3814329 thru 0x381a9b4 (0x668c bytes) 2024-09-21T22:51:08.253 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.269 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:08.268+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:08.276 INFO:teuthology.orchestra.run.smithi094.stdout:3581 writesame 0xd7cf180 thru 0xd7deb7d data_size 0xb(0xf9fe bytes) 2024-09-21T22:51:08.276 INFO:teuthology.orchestra.run.smithi094.stdout:3583 read 0xb0ae052 thru 0xb0b8689 (0xa638 bytes) 2024-09-21T22:51:08.281 INFO:teuthology.orchestra.run.smithi094.stdout:3584 trunc from 0xd7deb7e to 0xa0a77c6 2024-09-21T22:51:08.289 INFO:teuthology.orchestra.run.smithi094.stdout:3585 writesame 0x93231a1 thru 0x9326bf3 data_size 0x3f(0x3a53 bytes) 2024-09-21T22:51:08.297 INFO:teuthology.orchestra.run.smithi094.stdout:3586 punch from 0x7f1884 to 0x7fa802, (0x8f7e bytes) 2024-09-21T22:51:08.297 INFO:teuthology.orchestra.run.smithi094.stdout:3587 read 0x1d78f73 thru 0x1d83890 (0xa91e bytes) 2024-09-21T22:51:08.297 INFO:teuthology.orchestra.run.smithi094.stdout:3590 punch from 0x43d9dd7 to 0x43e1031, (0x725a bytes) 2024-09-21T22:51:08.297 INFO:teuthology.orchestra.run.smithi094.stdout:3591 read 0x6b259c2 thru 0x6b26cac (0x12eb bytes) 2024-09-21T22:51:08.306 INFO:teuthology.orchestra.run.smithi094.stdout:3592 writesame 0xa89557a thru 0xa8961d0 data_size 0x1b(0xc57 bytes) 2024-09-21T22:51:08.306 INFO:teuthology.orchestra.run.smithi094.stdout:3593 writesame 0xb77a1 thru 0xc39ca data_size 0x1d(0xc22a bytes) 2024-09-21T22:51:08.314 INFO:teuthology.orchestra.run.smithi094.stdout:3594 punch from 0x422a928 to 0x422d257, (0x292f bytes) 2024-09-21T22:51:08.314 INFO:teuthology.orchestra.run.smithi094.stdout:3595 write 0xa50b94f thru 0xa510d7a (0x542c bytes) 2024-09-21T22:51:08.338 INFO:teuthology.orchestra.run.smithi094.stdout:3597 write 0xd5ae52a thru 0xd5ae68e (0x165 bytes) 2024-09-21T22:51:08.340 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:08.340 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:08.344 INFO:teuthology.orchestra.run.smithi094.stdout:3598 write 0xc079a29 thru 0xc0848ee (0xaec6 bytes) 2024-09-21T22:51:08.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:08.362 INFO:teuthology.orchestra.run.smithi094.stdout:3600 write 0x79cc3f7 thru 0x79d4390 (0x7f9a bytes) 2024-09-21T22:51:08.375 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.375 INFO:teuthology.orchestra.run.smithi094.stdout:3604 read 0xc7d05b7 thru 0xc7d782d (0x7277 bytes) 2024-09-21T22:51:08.375 INFO:teuthology.orchestra.run.smithi094.stdout:3606 writesame 0x824459e thru 0x824948f data_size 0x2f(0x4ef2 bytes) 2024-09-21T22:51:08.384 INFO:teuthology.orchestra.run.smithi094.stdout:3607 punch from 0xc7b331a to 0xc7c2417, (0xf0fd bytes) 2024-09-21T22:51:08.384 INFO:teuthology.orchestra.run.smithi094.stdout:3608 write 0x485cbc4 thru 0x4866f45 (0xa382 bytes) 2024-09-21T22:51:08.399 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:08.396+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:08.410 INFO:teuthology.orchestra.run.smithi094.stdout:3609 write 0xd91b7dd thru 0xd926ceb (0xb50f bytes) 2024-09-21T22:51:08.426 INFO:teuthology.orchestra.run.smithi094.stdout:3610 read 0x10eff49 thru 0x10f6cea (0x6da2 bytes) 2024-09-21T22:51:08.426 INFO:teuthology.orchestra.run.smithi094.stdout:3611 write 0x84ef630 thru 0x84f4e08 (0x57d9 bytes) 2024-09-21T22:51:08.437 INFO:teuthology.orchestra.run.smithi094.stdout:3613 punch from 0x6ce7b9a to 0x6cf2731, (0xab97 bytes) 2024-09-21T22:51:08.437 INFO:teuthology.orchestra.run.smithi094.stdout:3614 write 0x1a560c9 thru 0x1a58988 (0x28c0 bytes) 2024-09-21T22:51:08.444 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.456 INFO:teuthology.orchestra.run.smithi094.stdout:3618 trunc from 0xd926cec to 0x591a607 2024-09-21T22:51:08.471 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:08.464+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:08.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:3619 trunc from 0x591a607 to 0x21739af 2024-09-21T22:51:08.529 INFO:teuthology.orchestra.run.smithi094.stdout:3620 write 0x3242579 thru 0x324fbaf (0xd637 bytes) 2024-09-21T22:51:08.535 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:08.535 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:08.549 INFO:teuthology.orchestra.run.smithi094.stdout:3621 punch from 0x22ecff0 to 0x22ee714, (0x1724 bytes) 2024-09-21T22:51:08.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:08.566 INFO:teuthology.orchestra.run.smithi094.stdout:3622 write 0x92bbb2d thru 0x92c1cb5 (0x6189 bytes) 2024-09-21T22:51:08.577 INFO:teuthology.orchestra.run.smithi094.stdout:3623 punch from 0x22fe0f3 to 0x2306283, (0x8190 bytes) 2024-09-21T22:51:08.578 INFO:teuthology.orchestra.run.smithi094.stdout:3624 read 0x7299ebc thru 0x72a98e6 (0xfa2b bytes) 2024-09-21T22:51:08.578 INFO:teuthology.orchestra.run.smithi094.stdout:3625 read 0x359fdd7 thru 0x35ac66d (0xc897 bytes) 2024-09-21T22:51:08.578 INFO:teuthology.orchestra.run.smithi094.stdout:3626 read 0x43169af thru 0x431d57a (0x6bcc bytes) 2024-09-21T22:51:08.590 INFO:teuthology.orchestra.run.smithi094.stdout:3628 write 0xd13dbe3 thru 0xd140989 (0x2da7 bytes) 2024-09-21T22:51:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:3629 read 0x6bcdc62 thru 0x6bd8420 (0xa7bf bytes) 2024-09-21T22:51:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:3630 read 0x3424a32 thru 0x3429aa2 (0x5071 bytes) 2024-09-21T22:51:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:3631 punch from 0x6cbfce9 to 0x6ccd767, (0xda7e bytes) 2024-09-21T22:51:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:3632 punch from 0x841de68 to 0x84240b7, (0x624f bytes) 2024-09-21T22:51:08.598 INFO:teuthology.orchestra.run.smithi094.stdout:3634 write 0xc165f82 thru 0xc169fb6 (0x4035 bytes) 2024-09-21T22:51:08.598 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:08.596+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:08.606 INFO:teuthology.orchestra.run.smithi094.stdout:3635 write 0xb01fad8 thru 0xb02c138 (0xc661 bytes) 2024-09-21T22:51:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:3639 writesame 0x86b48c thru 0x871978 data_size 0x1(0x64ed bytes) 2024-09-21T22:51:08.638 INFO:teuthology.orchestra.run.smithi094.stdout:3640 read 0x3a928f6 thru 0x3a9fbc7 (0xd2d2 bytes) 2024-09-21T22:51:08.647 INFO:teuthology.orchestra.run.smithi094.stdout:3641 writesame 0xe4cbd17 thru 0xe4cead3 data_size 0x3(0x2dbd bytes) 2024-09-21T22:51:08.659 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:3642 trunc from 0xe4cead4 to 0x6701df3 2024-09-21T22:51:08.673 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:08.668+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:08.681 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:08.681 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:08.691 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:3644 trunc from 0x6701df3 to 0x566a52 2024-09-21T22:51:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:08.710 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:08.728 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:08.728 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:08.730 INFO:teuthology.orchestra.run.smithi094.stdout:3646 trunc from 0x566a52 to 0xad40633 2024-09-21T22:51:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:3647 write 0x695cfc3 thru 0x695fa10 (0x2a4e bytes) 2024-09-21T22:51:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:3650 punch from 0x456aac0 to 0x4571869, (0x6da9 bytes) 2024-09-21T22:51:08.747 INFO:teuthology.orchestra.run.smithi094.stdout:3651 trunc from 0xad40633 to 0xc1cb922 2024-09-21T22:51:08.749 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:08.750 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:3652 punch from 0x6c8786a to 0x6c951b6, (0xd94c bytes) 2024-09-21T22:51:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:3653 punch from 0x3ec6b8d to 0x3ec7e47, (0x12ba bytes) 2024-09-21T22:51:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:3654 read 0x5a5dc8d thru 0x5a685a6 (0xa91a bytes) 2024-09-21T22:51:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:3656 trunc from 0xc1cb922 to 0x94e51fa 2024-09-21T22:51:08.769 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:08.769 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:08.774 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:08.768+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:08.777 INFO:teuthology.orchestra.run.smithi094.stdout:3657 write 0xd6d697a thru 0xd6dc766 (0x5ded bytes) 2024-09-21T22:51:08.787 INFO:teuthology.orchestra.run.smithi094.stdout:3658 read 0x7a9a99a thru 0x7aa4a4b (0xa0b2 bytes) 2024-09-21T22:51:08.787 INFO:teuthology.orchestra.run.smithi094.stdout:3659 write 0xafc2e49 thru 0xafd04e1 (0xd699 bytes) 2024-09-21T22:51:08.791 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:08.802 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.802 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.803 INFO:teuthology.orchestra.run.smithi094.stdout:3662 write 0x5fec24a thru 0x5ff9355 (0xd10c bytes) 2024-09-21T22:51:08.818 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:08.819 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:08.819 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.819 INFO:teuthology.orchestra.run.smithi094.stdout:3666 read 0x45644f7 thru 0x4572469 (0xdf73 bytes) 2024-09-21T22:51:08.820 INFO:teuthology.orchestra.run.smithi094.stdout:3667 writesame 0xb7bd251 thru 0xb7c8f4b data_size 0x1(0xbcfb bytes) 2024-09-21T22:51:08.841 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:08.842 INFO:teuthology.orchestra.run.smithi094.stdout:3670 punch from 0x13f5587 to 0x13f75ea, (0x2063 bytes) 2024-09-21T22:51:08.843 INFO:teuthology.orchestra.run.smithi094.stdout:3671 trunc from 0xd6dc767 to 0xc30170d 2024-09-21T22:51:08.858 INFO:teuthology.orchestra.run.smithi094.stdout:3672 punch from 0x9f2ca41 to 0x9f39542, (0xcb01 bytes) 2024-09-21T22:51:08.858 INFO:teuthology.orchestra.run.smithi094.stdout:3673 clone 22 order 24 su 8388608 sc 6 2024-09-21T22:51:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:08.875 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:08.868+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:08.877 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:08.976 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:08.972+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:08.997 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:08.997 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:08.998 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:08.998 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:08.998 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:09.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:09.076 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:09.072+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:09.135 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.196301789000003, 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:09.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:09.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:09.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:09.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:09.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:09.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37718 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.651294314000001, 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096520755999999999, 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37718 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480304+0000", 2024-09-21T22:51:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480305+0000", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3699999999999999e-07 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480310+0000", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480328+0000", 2024-09-21T22:51:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8272000000000001e-05 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480342+0000", 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3776e-05 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576157+0000", 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.095814472999999997 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576347+0000", 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019016200000000001 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576825+0000", 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047779300000000001 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.65122993, 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.651166480000001, 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.651103301999999, 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.651046961999999, 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.651005220000002, 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650944619000001, 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650864182999999, 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650788073000001, 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650701000000002, 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650657511999999, 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650623874000001, 2024-09-21T22:51:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650590107999999, 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650558824000001, 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650513469, 2024-09-21T22:51:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650481973000002, 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.650447192000001, 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.632501660999999, 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.632440617, 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.224 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:09.225 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:51:09.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:09.226 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:09.224+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:09.326 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:51:09.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T22:51:09.327 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:09.320+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:09.400 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:09.401 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:09.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:09.454 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T22:51:09.456 INFO:teuthology.orchestra.run.smithi094.stdout:3675 write 0x224341a thru 0x224407e (0xc65 bytes) 2024-09-21T22:51:09.461 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:09.460+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:09.465 INFO:teuthology.orchestra.run.smithi094.stdout:3676 write 0x5230965 thru 0x52398af (0x8f4b bytes) 2024-09-21T22:51:09.468 INFO:teuthology.orchestra.run.smithi094.stdout:3678 write 0x8eaebd9 thru 0x8eb132f (0x2757 bytes) 2024-09-21T22:51:09.475 INFO:teuthology.orchestra.run.smithi094.stdout:3679 writesame 0x81f77d6 thru 0x8204d92 data_size 0x3(0xd5bd bytes) 2024-09-21T22:51:09.492 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:09.500 INFO:teuthology.orchestra.run.smithi094.stdout:3682 trunc from 0xc30170d to 0x6aed824 2024-09-21T22:51:09.522 INFO:teuthology.orchestra.run.smithi094.stdout:3683 punch from 0x49949d5 to 0x499cf22, (0x854d bytes) 2024-09-21T22:51:09.527 INFO:teuthology.orchestra.run.smithi094.stdout:3684 trunc from 0x6aed824 to 0x304a971 2024-09-21T22:51:09.528 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:09.524+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:09.549 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:09.549 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:09.550 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:09.550 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:09.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:09.592 INFO:teuthology.orchestra.run.smithi094.stdout:3685 punch from 0x149b16a to 0x14a829f, (0xd135 bytes) 2024-09-21T22:51:09.592 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:09.617 INFO:teuthology.orchestra.run.smithi094.stdout:3687 write 0xa71b4b1 thru 0xa71b5a6 (0xf6 bytes) 2024-09-21T22:51:09.622 INFO:teuthology.orchestra.run.smithi094.stdout:3688 write 0x4c2b225 thru 0x4c2f791 (0x456d bytes) 2024-09-21T22:51:09.626 INFO:teuthology.orchestra.run.smithi094.stdout:3689 punch from 0x8221c13 to 0x822489d, (0x2c8a bytes) 2024-09-21T22:51:09.628 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:09.624+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:09.645 INFO:teuthology.orchestra.run.smithi094.stdout:3690 trunc from 0xa71b5a7 to 0x138321e 2024-09-21T22:51:09.674 INFO:teuthology.orchestra.run.smithi094.stdout:3691 read 0xd47d83 thru 0xd51378 (0x95f6 bytes) 2024-09-21T22:51:09.675 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:09.675 INFO:teuthology.orchestra.run.smithi094.stdout:3693 read 0x740f80 thru 0x7489a8 (0x7a29 bytes) 2024-09-21T22:51:09.676 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:09.676 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:09.676 INFO:teuthology.orchestra.run.smithi094.stdout:3697 read 0xb81a50 thru 0xb84919 (0x2eca bytes) 2024-09-21T22:51:09.676 INFO:teuthology.orchestra.run.smithi094.stdout:3698 clone 23 order 22 su 65536 sc 11 2024-09-21T22:51:09.729 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:09.724+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:09.733 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.711062859000002, 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.706825289999998, 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.706630392000001, 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68484 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 356352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0920836549999997, 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033448665000000002, 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68484 2024-09-21T22:51:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638316+0000", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0100000000000001e-06 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638322+0000", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169999999999997e-06 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638342+0000", 2024-09-21T22:51:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.995e-05 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638370+0000", 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7985000000000001e-05 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671391+0000", 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033021259999999997 2024-09-21T22:51:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671483+0000", 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1380999999999998e-05 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671763+0000", 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028066199999999999 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68541 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0920079469999999, 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033895258999999997, 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68541 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638391+0000", 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638398+0000", 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9770000000000003e-06 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638421+0000", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2578999999999998e-05 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638436+0000", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4620000000000001e-05 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671874+0000", 2024-09-21T22:51:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033438273999999997 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671995+0000", 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120567 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672286+0000", 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029103199999999999 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68542 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 49152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0919433490000001, 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034317184000000001, 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68542 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638456+0000", 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.567e-06 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638462+0000", 2024-09-21T22:51:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9830000000000001e-06 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638487+0000", 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4199999999999999e-05 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638501+0000", 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4489e-05 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672333+0000", 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033832167000000003 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672439+0000", 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105635 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672772+0000", 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033314299999999999 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68543 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.09187806, 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034634933999999999, 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68543 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638521+0000", 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1060000000000001e-06 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638527+0000", 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638546+0000", 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8635999999999999e-05 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638559+0000", 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2733e-05 2024-09-21T22:51:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672814+0000", 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034255173 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672909+0000", 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4581999999999997e-05 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673155+0000", 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024655400000000001 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68544 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 163840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0918207249999998, 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034985167999999997, 2024-09-21T22:51:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68544 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638579+0000", 2024-09-21T22:51:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006e-06 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638584+0000", 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6840000000000001e-06 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638604+0000", 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9859000000000001e-05 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638618+0000", 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.428e-05 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673193+0000", 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034575074999999997 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673252+0000", 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8288000000000002e-05 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673563+0000", 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031097600000000003 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68545 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 196608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0916818560000001, 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035408569000000001, 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68545 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638718+0000", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638725+0000", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8299999999999998e-06 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638746+0000", 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1566e-05 2024-09-21T22:51:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638760+0000", 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673630+0000", 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034869372000000003 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673803+0000", 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000173435 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674125+0000", 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032173299999999999 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68546 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 245760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0916057710000002, 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035763809000000001, 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68546 2024-09-21T22:51:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638794+0000", 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4529999999999999e-06 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638815+0000", 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1418999999999999e-05 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638837+0000", 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1855999999999999e-05 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638851+0000", 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.341e-05 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674168+0000", 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035317861999999998 2024-09-21T22:51:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674234+0000", 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5946999999999997e-05 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674556+0000", 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032186200000000002 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68547 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 278528~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0915252339999997, 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03614469, 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68547 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638874+0000", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638881+0000", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9999999999999999e-06 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638901+0000", 2024-09-21T22:51:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999999e-05 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638915+0000", 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4532000000000001e-05 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674611+0000", 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035696033000000002 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674712+0000", 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010040299999999999 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675018+0000", 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030589199999999999 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68548 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0914645089999997, 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036468652999999997, 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68548 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638935+0000", 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638941+0000", 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638959+0000", 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8431999999999999e-05 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638973+0000", 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3813e-05 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675083+0000", 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036110541000000003 2024-09-21T22:51:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675199+0000", 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115926 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675402+0000", 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020310400000000001 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68605 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0914073310000001, 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036815352000000003, 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68605 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638992+0000", 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638998+0000", 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4370000000000004e-06 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639017+0000", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639055+0000", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7441999999999998e-05 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675453+0000", 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036398463999999998 2024-09-21T22:51:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675531+0000", 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7575000000000002e-05 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675806+0000", 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027533800000000003 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68606 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0913226260000002, 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036991366999999997, 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68606 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639077+0000", 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639085+0000", 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7559999999999999e-06 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639118+0000", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3705999999999999e-05 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639148+0000", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0162999999999999e-05 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675849+0000", 2024-09-21T22:51:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036700993000000001 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675907+0000", 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8032999999999998e-05 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676067+0000", 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015953300000000001 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68607 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 86016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0912286089999998, 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037187378, 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68607 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639171+0000", 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639178+0000", 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6699999999999997e-06 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639198+0000", 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0132e-05 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639212+0000", 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4294000000000001e-05 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676124+0000", 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036912568 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676193+0000", 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8821999999999999e-05 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676357+0000", 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016368200000000001 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68608 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0911645979999998, 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037593025000000002, 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68608 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639235+0000", 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.034e-06 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639241+0000", 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5830000000000003e-06 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639268+0000", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6741999999999999e-05 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639281+0000", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.344e-05 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676415+0000", 2024-09-21T22:51:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037133806999999998 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676541+0000", 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012614500000000001 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676827+0000", 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028527400000000002 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68609 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0910973830000001, 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037858175000000001, 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68609 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639302+0000", 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0669999999999999e-06 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639309+0000", 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7429999999999996e-06 2024-09-21T22:51:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639327+0000", 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8573e-05 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639341+0000", 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3286e-05 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676887+0000", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037546698000000003 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676966+0000", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8611999999999999e-05 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677159+0000", 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000193196 2024-09-21T22:51:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68610 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0910389340000002, 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038202400999999997, 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68610 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639360+0000", 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639367+0000", 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1759999999999998e-06 2024-09-21T22:51:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639385+0000", 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8454e-05 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639399+0000", 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3835999999999999e-05 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677208+0000", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037809295999999999 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677317+0000", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108682 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677562+0000", 2024-09-21T22:51:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024489100000000001 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68611 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0909804660000004, 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038561235999999999, 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68611 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639419+0000", 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639425+0000", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639443+0000", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8675999999999999e-05 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639457+0000", 2024-09-21T22:51:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4083000000000001e-05 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677605+0000", 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038147542999999999 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677713+0000", 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108529 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677979+0000", 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026561500000000002 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68612 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0908922360000002, 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038912731999999998, 2024-09-21T22:51:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68612 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639508+0000", 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639514+0000", 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3500000000000002e-06 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639534+0000", 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0199000000000001e-05 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639549+0000", 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4056000000000001e-05 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678034+0000", 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038485761 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678166+0000", 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131719 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678419+0000", 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025270300000000001 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:09.842 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:09.844 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:09.840+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:09.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:09.944 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:09.940+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:09.977 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:09.977 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:09.977 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:09.977 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:09.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:10.045 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:10.044+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:10.113 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:10.113 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:10.113 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:10.113 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:10.113 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:10.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:10.146 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:10.144+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T22:51:10.246 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:10.244+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:10.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.777403983999999, 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:10.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:51:10.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.777364353999999, 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:51:10.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.776598869000001, 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:51:10.267 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:51:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.776384921000002, 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:51:10.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:51:10.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:51:10.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.776294927999999, 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:51:10.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:51:10.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.776233712, 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:51:10.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:51:10.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215392342000001, 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:51:10.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:51:10.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215378794999999, 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:51:10.289 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:51:10.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215358413000001, 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:51:10.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:51:10.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:51:10.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215344984000001, 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:51:10.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:51:10.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215330520999998, 2024-09-21T22:51:10.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215317895999998, 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:51:10.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215279496000001, 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:51:10.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:51:10.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215255024000001, 2024-09-21T22:51:10.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:51:10.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215236159, 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.320 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:51:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215222921999999, 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:51:10.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:10.328 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215209948999998, 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:51:10.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215197154999998, 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.215184654000002, 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:51:10.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.214874227999999, 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.342 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:51:10.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:51:10.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.346 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:10.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:10.347 INFO:teuthology.orchestra.run.smithi094.stdout:leaving image image_client.0-clone22 intact 2024-09-21T22:51:10.348 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:10.344+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:10.404 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T22:51:10.448 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:10.440+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:10.518 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:10.519 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:10.519 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:10.519 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:10.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:10.548 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:10.544+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:10.649 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:10.644+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:10.687 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:10.687 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:10.688 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:10.688 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:10.688 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:10.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:10.750 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:10.748+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:10.781 INFO:teuthology.orchestra.run.smithi094.stdout:3699 punch from 0x106d772 to 0x1070847, (0x30d5 bytes) 2024-09-21T22:51:10.811 INFO:teuthology.orchestra.run.smithi094.stdout:3700 write 0x723e1dd thru 0x723edbd (0xbe1 bytes) 2024-09-21T22:51:10.818 INFO:teuthology.orchestra.run.smithi094.stdout:3703 read 0x67635cf thru 0x6772ab0 (0xf4e2 bytes) 2024-09-21T22:51:10.821 INFO:teuthology.orchestra.run.smithi094.stdout:3704 trunc from 0x723edbe to 0x8a9c681 2024-09-21T22:51:10.834 INFO:teuthology.orchestra.run.smithi094.stdout:3705 read 0x826ba40 thru 0x8278132 (0xc6f3 bytes) 2024-09-21T22:51:10.834 INFO:teuthology.orchestra.run.smithi094.stdout:3706 read 0x8a22b thru 0x936e0 (0x94b6 bytes) 2024-09-21T22:51:10.836 INFO:teuthology.orchestra.run.smithi094.stdout:3707 write 0x484164e thru 0x4844a56 (0x3409 bytes) 2024-09-21T22:51:10.843 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:10.843 INFO:teuthology.orchestra.run.smithi094.stdout:3709 punch from 0x5c3d0b2 to 0x5c4794d, (0xa89b bytes) 2024-09-21T22:51:10.843 INFO:teuthology.orchestra.run.smithi094.stdout:3710 write 0x388ba9b thru 0x38921d6 (0x673c bytes) 2024-09-21T22:51:10.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:10.850 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:10.848+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:10.852 INFO:teuthology.orchestra.run.smithi094.stdout:3711 read 0x6757434 thru 0x6765830 (0xe3fd bytes) 2024-09-21T22:51:10.852 INFO:teuthology.orchestra.run.smithi094.stdout:3712 punch from 0x3962e0c to 0x396cfd0, (0xa1c4 bytes) 2024-09-21T22:51:10.866 INFO:teuthology.orchestra.run.smithi094.stdout:3713 writesame 0x9e8984a thru 0x9e93739 data_size 0x8(0x9ef0 bytes) 2024-09-21T22:51:10.868 INFO:teuthology.orchestra.run.smithi094.stdout:3714 trunc from 0x9e9373a to 0xb546f49 2024-09-21T22:51:10.874 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:10.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 25.783765011, 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.875 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:10.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:10.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:10.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34471 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 25.778590178999998, 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.039490556000000003, 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.879 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34471 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093355+0000", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9800000000000004e-07 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093357+0000", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5210000000000001e-06 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093365+0000", 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093370+0000", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1889999999999998e-06 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128443+0000", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035073159999999999 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128522+0000", 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9128000000000005e-05 2024-09-21T22:51:10.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.132845+0000", 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043228559999999999 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.383565145999999, 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.887 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.382050885999998, 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.891 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381997584000001, 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381942536, 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381873351999999, 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.900 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:10.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:10.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381810784999999, 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.908 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381761391000001, 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381257607999999, 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.916 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.381190028999999, 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:10.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380640786000001, 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380563304999999, 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380399174000001, 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380338195, 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380294583000001, 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380250101000001, 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380204933000002, 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.380162636000001, 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.894730046999999, 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:10.961 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:10.962 INFO:teuthology.orchestra.run.smithi094.stdout:3715 punch from 0x92937f0 to 0x92a0237, (0xca47 bytes) 2024-09-21T22:51:10.962 INFO:teuthology.orchestra.run.smithi094.stdout:3716 writesame 0xa1e1fb1 thru 0xa1eafbc data_size 0x1c(0x900c bytes) 2024-09-21T22:51:10.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3717 write 0xc2f373d thru 0xc301d7f (0xe643 bytes) 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3718 read 0xa455f1b thru 0xa45bfa3 (0x6089 bytes) 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3721 trunc from 0xc301d80 to 0x7180e6e 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3722 read 0x17d0bb2 thru 0x17de9bb (0xde0a bytes) 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3723 read 0x5293bcf thru 0x5298785 (0x4bb7 bytes) 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3724 read 0x154faf9 thru 0x15555e7 (0x5aef bytes) 2024-09-21T22:51:10.964 INFO:teuthology.orchestra.run.smithi094.stdout:3725 write 0x8cf9234 thru 0x8cffc3f (0x6a0c bytes) 2024-09-21T22:51:10.964 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:10.956+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:10.969 INFO:teuthology.orchestra.run.smithi094.stdout:3726 punch from 0x4d45a7f to 0x4d47ad5, (0x2056 bytes) 2024-09-21T22:51:10.969 INFO:teuthology.orchestra.run.smithi094.stdout:3728 write 0x78d3118 thru 0x78dd70b (0xa5f4 bytes) 2024-09-21T22:51:10.975 INFO:teuthology.orchestra.run.smithi094.stdout:3730 punch from 0x25b5437 to 0x25b712d, (0x1cf6 bytes) 2024-09-21T22:51:10.975 INFO:teuthology.orchestra.run.smithi094.stdout:3731 read 0x167a82d thru 0x1684352 (0x9b26 bytes) 2024-09-21T22:51:10.975 INFO:teuthology.orchestra.run.smithi094.stdout:3732 read 0x41f3f7c thru 0x41f897d (0x4a02 bytes) 2024-09-21T22:51:10.975 INFO:teuthology.orchestra.run.smithi094.stdout:3733 writesame 0x40b9fb7 thru 0x40bf013 data_size 0x1(0x505d bytes) 2024-09-21T22:51:10.987 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:10.987 INFO:teuthology.orchestra.run.smithi094.stdout:3735 punch from 0x1bf320b to 0x1c01cf5, (0xeaea bytes) 2024-09-21T22:51:10.987 INFO:teuthology.orchestra.run.smithi094.stdout:3736 punch from 0x3ca1c88 to 0x3ca5213, (0x358b bytes) 2024-09-21T22:51:10.987 INFO:teuthology.orchestra.run.smithi094.stdout:3737 writesame 0x8d9c41 thru 0x8e381c data_size 0x3c(0x9bdc bytes) 2024-09-21T22:51:10.996 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:10.996 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:11.013 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:11.017 INFO:teuthology.orchestra.run.smithi094.stdout:3738 write 0x708d2fc thru 0x7092a44 (0x5749 bytes) 2024-09-21T22:51:11.034 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:11.034 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:11.043 INFO:teuthology.orchestra.run.smithi094.stdout:3739 write 0xbfb7901 thru 0xbfb92fd (0x19fd bytes) 2024-09-21T22:51:11.051 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:11.051 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:11.063 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:11.060+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:11.064 INFO:teuthology.orchestra.run.smithi094.stdout:3740 write 0xdbb2d9f thru 0xdbb803a (0x529c bytes) 2024-09-21T22:51:11.070 INFO:teuthology.orchestra.run.smithi094.stdout:3742 write 0x224fd45 thru 0x225e8ca (0xeb86 bytes) 2024-09-21T22:51:11.072 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:3745 writesame 0x9cbae8a thru 0x9cbdbee data_size 0x1(0x2d65 bytes) 2024-09-21T22:51:11.085 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.085 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.093 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:11.094 INFO:teuthology.orchestra.run.smithi094.stdout:3746 read 0x6027deb thru 0x60370b0 (0xf2c6 bytes) 2024-09-21T22:51:11.094 INFO:teuthology.orchestra.run.smithi094.stdout:3747 writesame 0xa916681 thru 0xa919716 data_size 0x9(0x3096 bytes) 2024-09-21T22:51:11.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:11.098 INFO:teuthology.orchestra.run.smithi094.stdout:3750 write 0x54d6633 thru 0x54e3ddb (0xd7a9 bytes) 2024-09-21T22:51:11.115 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:11.122 INFO:teuthology.orchestra.run.smithi094.stdout:3752 write 0xebbe6c4 thru 0xebcc9ba (0xe2f7 bytes) 2024-09-21T22:51:11.131 INFO:teuthology.orchestra.run.smithi094.stdout:3755 read 0x1662dd6 thru 0x166c7df (0x9a0a bytes) 2024-09-21T22:51:11.132 INFO:teuthology.orchestra.run.smithi094.stdout:3756 punch from 0xe41c11b to 0xe41c475, (0x35a bytes) 2024-09-21T22:51:11.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:11.138 INFO:teuthology.orchestra.run.smithi094.stdout:3757 trunc from 0xebcc9bb to 0x99a70d4 2024-09-21T22:51:11.163 INFO:teuthology.orchestra.run.smithi094.stdout:3758 read 0x8708ccc thru 0x870f0ad (0x63e2 bytes) 2024-09-21T22:51:11.164 INFO:teuthology.orchestra.run.smithi094.stdout:3760 writesame 0x950c9dc thru 0x95188c4 data_size 0xb(0xbee9 bytes) 2024-09-21T22:51:11.164 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:11.156+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:11.196 INFO:teuthology.orchestra.run.smithi094.stdout:3762 write 0xcbd0193 thru 0xcbdfe5d (0xfccb bytes) 2024-09-21T22:51:11.204 INFO:teuthology.orchestra.run.smithi094.stdout:3763 write 0x34c2c4d thru 0x34cd58c (0xa940 bytes) 2024-09-21T22:51:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:3764 writesame 0x5e8cf45 thru 0x5e94ce0 data_size 0x2(0x7d9c bytes) 2024-09-21T22:51:11.216 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.216 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.223 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.223 INFO:teuthology.orchestra.run.smithi094.stdout:3766 write 0x90c33af thru 0x90c584b (0x249d bytes) 2024-09-21T22:51:11.229 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T22:51:11.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:11.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:51:11.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T22:51:11.249 INFO:teuthology.orchestra.run.smithi094.stdout:3767 writesame 0xde4af1b thru 0xde587ee data_size 0x2(0xd8d4 bytes) 2024-09-21T22:51:11.249 INFO:teuthology.orchestra.run.smithi094.stdout:3769 punch from 0x1de1526 to 0x1dee1c1, (0xcc9b bytes) 2024-09-21T22:51:11.250 INFO:teuthology.orchestra.run.smithi094.stdout:3770 read 0x3a33816 thru 0x3a417db (0xdfc6 bytes) 2024-09-21T22:51:11.251 INFO:teuthology.orchestra.run.smithi094.stdout:3771 punch from 0x7003fdd to 0x70132f4, (0xf317 bytes) 2024-09-21T22:51:11.251 INFO:teuthology.orchestra.run.smithi094.stdout:3772 punch from 0xdc61144 to 0xdc67f6d, (0x6e29 bytes) 2024-09-21T22:51:11.251 INFO:teuthology.orchestra.run.smithi094.stdout:3774 write 0xb62a387 thru 0xb62f2f6 (0x4f70 bytes) 2024-09-21T22:51:11.258 INFO:teuthology.orchestra.run.smithi094.stdout:3775 read 0x8ce63ce thru 0x8ceb8ff (0x5532 bytes) 2024-09-21T22:51:11.259 INFO:teuthology.orchestra.run.smithi094.stdout:3776 writesame 0xa8cc8c2 thru 0xa8d48f5 data_size 0x1e(0x8034 bytes) 2024-09-21T22:51:11.264 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:11.260+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:11.276 INFO:teuthology.orchestra.run.smithi094.stdout:3777 trunc from 0xde587ef to 0x56051ae 2024-09-21T22:51:11.311 INFO:teuthology.orchestra.run.smithi094.stdout:3778 writesame 0x340c15d thru 0x341bea3 data_size 0x1(0xfd47 bytes) 2024-09-21T22:51:11.346 INFO:teuthology.orchestra.run.smithi094.stdout:3779 trunc from 0x56051ae to 0x98f70e0 2024-09-21T22:51:11.361 INFO:teuthology.orchestra.run.smithi094.stdout:3780 trunc from 0x98f70e0 to 0xb2705f5 2024-09-21T22:51:11.365 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:11.364+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:11.372 INFO:teuthology.orchestra.run.smithi094.stdout:3782 read 0x65f1835 thru 0x65fc49d (0xac69 bytes) 2024-09-21T22:51:11.377 INFO:teuthology.orchestra.run.smithi094.stdout:3784 trunc from 0xb2705f5 to 0xebdac79 2024-09-21T22:51:11.389 INFO:teuthology.orchestra.run.smithi094.stdout:3786 writesame 0xd3c954c thru 0xd3d74ab data_size 0x10(0xdf60 bytes) 2024-09-21T22:51:11.397 INFO:teuthology.orchestra.run.smithi094.stdout:3788 punch from 0x445c70b to 0x446c5e4, (0xfed9 bytes) 2024-09-21T22:51:11.403 INFO:teuthology.orchestra.run.smithi094.stdout:3791 trunc from 0xebdac79 to 0xb2e0e94 2024-09-21T22:51:11.404 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.404 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:11.421 INFO:teuthology.orchestra.run.smithi094.stdout:3793 trunc from 0xb2e0e94 to 0xcceafd0 2024-09-21T22:51:11.433 INFO:teuthology.orchestra.run.smithi094.stdout:3794 read 0x9e2ab4 thru 0x9e993f (0x6e8c bytes) 2024-09-21T22:51:11.447 INFO:teuthology.orchestra.run.smithi094.stdout:3795 write 0xdfd9357 thru 0xdfe8025 (0xeccf bytes) 2024-09-21T22:51:11.458 INFO:teuthology.orchestra.run.smithi094.stdout:3796 read 0xd47631a thru 0xd480aef (0xa7d6 bytes) 2024-09-21T22:51:11.458 INFO:teuthology.orchestra.run.smithi094.stdout:3797 read 0x9cd1922 thru 0x9cd901e (0x76fd bytes) 2024-09-21T22:51:11.466 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:11.464+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:11.476 INFO:teuthology.orchestra.run.smithi094.stdout:3799 trunc from 0xdfe8026 to 0x2249d8e 2024-09-21T22:51:11.539 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.540 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:11.567 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:11.564+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:3800 read 0x14834b6 thru 0x1486fc0 (0x3b0b bytes) 2024-09-21T22:51:11.605 INFO:teuthology.orchestra.run.smithi094.stdout:3801 punch from 0x12c411 to 0x12fb72, (0x3761 bytes) 2024-09-21T22:51:11.605 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.629 INFO:teuthology.orchestra.run.smithi094.stdout:3803 write 0xb591144 thru 0xb59eddc (0xdc99 bytes) 2024-09-21T22:51:11.634 INFO:teuthology.orchestra.run.smithi094.stdout:3805 writesame 0x37dc7a1 thru 0x37e6f76 data_size 0xba(0xa7d6 bytes) 2024-09-21T22:51:11.643 INFO:teuthology.orchestra.run.smithi094.stdout:3806 write 0xa9e4e0b thru 0xa9ed920 (0x8b16 bytes) 2024-09-21T22:51:11.650 INFO:teuthology.orchestra.run.smithi094.stdout:3807 read 0x2fe26ec thru 0x2fe95e8 (0x6efd bytes) 2024-09-21T22:51:11.655 INFO:teuthology.orchestra.run.smithi094.stdout:3808 trunc from 0xb59eddd to 0x79e96d2 2024-09-21T22:51:11.667 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:11.660+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:11.677 INFO:teuthology.orchestra.run.smithi094.stdout:3809 read 0x311d2ed thru 0x311f7d5 (0x24e9 bytes) 2024-09-21T22:51:11.677 INFO:teuthology.orchestra.run.smithi094.stdout:3810 read 0x62c15cb thru 0x62c8936 (0x736c bytes) 2024-09-21T22:51:11.683 INFO:teuthology.orchestra.run.smithi094.stdout:3811 trunc from 0x79e96d2 to 0x490710b 2024-09-21T22:51:11.698 INFO:teuthology.orchestra.run.smithi094.stdout:3812 read 0x219dc5d thru 0x21a1a44 (0x3de8 bytes) 2024-09-21T22:51:11.700 INFO:teuthology.orchestra.run.smithi094.stdout:3813 write 0x474881c thru 0x474a8c4 (0x20a9 bytes) 2024-09-21T22:51:11.706 INFO:teuthology.orchestra.run.smithi094.stdout:3814 read 0x472485f thru 0x472a2f0 (0x5a92 bytes) 2024-09-21T22:51:11.711 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.711 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.712 INFO:teuthology.orchestra.run.smithi094.stdout:3817 trunc from 0x490710b to 0x576237 2024-09-21T22:51:11.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:11.750 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.768 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:11.764+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:11.780 INFO:teuthology.orchestra.run.smithi094.stdout:3819 write 0xb743a50 thru 0xb7537e6 (0xfd97 bytes) 2024-09-21T22:51:11.792 INFO:teuthology.orchestra.run.smithi094.stdout:3820 read 0x846e08e thru 0x84775da (0x954d bytes) 2024-09-21T22:51:11.793 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.793 INFO:teuthology.orchestra.run.smithi094.stdout:3822 writesame 0x5e6eca8 thru 0x5e76236 data_size 0xd(0x758f bytes) 2024-09-21T22:51:11.802 INFO:teuthology.orchestra.run.smithi094.stdout:3823 read 0xb6c8a22 thru 0xb6c90e0 (0x6bf bytes) 2024-09-21T22:51:11.802 INFO:teuthology.orchestra.run.smithi094.stdout:3824 writesame 0x9f70916 thru 0x9f78741 data_size 0x64(0x7e2c bytes) 2024-09-21T22:51:11.828 INFO:teuthology.orchestra.run.smithi094.stdout:3825 write 0xecb3236 thru 0xecb4c00 (0x19cb bytes) 2024-09-21T22:51:11.836 INFO:teuthology.orchestra.run.smithi094.stdout:3826 punch from 0x3f200c0 to 0x3f23c1f, (0x3b5f bytes) 2024-09-21T22:51:11.843 INFO:teuthology.orchestra.run.smithi094.stdout:3827 trunc from 0xecb4c01 to 0xb7ec5f5 2024-09-21T22:51:11.865 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:11.865 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:11.869 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:11.864+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:11.871 INFO:teuthology.orchestra.run.smithi094.stdout:3828 write 0x5776083 thru 0x577a7d2 (0x4750 bytes) 2024-09-21T22:51:11.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:11.879 INFO:teuthology.orchestra.run.smithi094.stdout:3830 read 0x340925 thru 0x342244 (0x1920 bytes) 2024-09-21T22:51:11.886 INFO:teuthology.orchestra.run.smithi094.stdout:3832 trunc from 0xb7ec5f5 to 0xe80b91e 2024-09-21T22:51:11.900 INFO:teuthology.orchestra.run.smithi094.stdout:3833 write 0x2e27c17 thru 0x2e32b5c (0xaf46 bytes) 2024-09-21T22:51:11.928 INFO:teuthology.orchestra.run.smithi094.stdout:3835 trunc from 0xe80b91e to 0x3002670 2024-09-21T22:51:11.955 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:11.955 INFO:teuthology.orchestra.run.smithi094.stdout:3839 read 0x1d6394d thru 0x1d6c8fc (0x8fb0 bytes) 2024-09-21T22:51:11.956 INFO:teuthology.orchestra.run.smithi094.stdout:3841 read 0x223ba2 thru 0x22db7f (0x9fde bytes) 2024-09-21T22:51:11.969 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:11.964+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:11.987 INFO:teuthology.orchestra.run.smithi094.stdout:3842 writesame 0xb4988fc thru 0xb4a6aed data_size 0x1(0xe1f2 bytes) 2024-09-21T22:51:12.012 INFO:teuthology.orchestra.run.smithi094.stdout:3843 trunc from 0xb4a6aee to 0xb4a919 2024-09-21T22:51:12.012 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.013 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:12.039 INFO:teuthology.orchestra.run.smithi094.stdout:3844 read 0x4fb297 thru 0x505395 (0xa0ff bytes) 2024-09-21T22:51:12.040 INFO:teuthology.orchestra.run.smithi094.stdout:3845 read 0x6556c6 thru 0x65e4f6 (0x8e31 bytes) 2024-09-21T22:51:12.052 INFO:teuthology.orchestra.run.smithi094.stdout:3846 write 0x11534e6 thru 0x1162d77 (0xf892 bytes) 2024-09-21T22:51:12.055 INFO:teuthology.orchestra.run.smithi094.stdout:3848 punch from 0x121a10 to 0x123c47, (0x2237 bytes) 2024-09-21T22:51:12.056 INFO:teuthology.orchestra.run.smithi094.stdout:3849 read 0x7d16ad thru 0x7dc5b6 (0xaf0a bytes) 2024-09-21T22:51:12.056 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:12.056 INFO:teuthology.orchestra.run.smithi094.stdout:3851 punch from 0xe19ec to 0xeb588, (0x9b9c bytes) 2024-09-21T22:51:12.056 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:12.069 INFO:teuthology.orchestra.run.smithi094.stdout:3854 write 0x29f5879 thru 0x29ffdc6 (0xa54e bytes) 2024-09-21T22:51:12.070 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:12.064+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:12.072 INFO:teuthology.orchestra.run.smithi094.stdout:3856 clone 24 order 18 su 2048 sc 11 2024-09-21T22:51:12.151 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.151 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:12.171 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:12.164+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:51:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T22:51:12.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T22:51:12.271 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:12.268+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:12.297 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.298 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:12.356 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone23 from 0x29ffdc7 (overlap 0x576237) to 0x346e21 2024-09-21T22:51:12.398 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:12.396+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:12.452 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T22:51:12.473 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:12.468+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:12.489 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.490 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:12.508 INFO:teuthology.orchestra.run.smithi094.stdout:3858 write 0x7819a3d thru 0x7822d5d (0x9321 bytes) 2024-09-21T22:51:12.534 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:12.552 INFO:teuthology.orchestra.run.smithi094.stdout:3860 write 0xd7908d7 thru 0xd791f3c (0x1666 bytes) 2024-09-21T22:51:12.560 INFO:teuthology.orchestra.run.smithi094.stdout:3861 write 0x6d08c94 thru 0x6d1447d (0xb7ea bytes) 2024-09-21T22:51:12.584 INFO:teuthology.orchestra.run.smithi094.stdout:3862 read 0x47d475b thru 0x47daed0 (0x6776 bytes) 2024-09-21T22:51:12.584 INFO:teuthology.orchestra.run.smithi094.stdout:3863 writesame 0x1ab9d18 thru 0x1abc3fb data_size 0x26(0x26e4 bytes) 2024-09-21T22:51:12.593 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:12.588+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:12.643 INFO:teuthology.orchestra.run.smithi094.stdout:3864 read 0x730e9e3 thru 0x7313112 (0x4730 bytes) 2024-09-21T22:51:12.643 INFO:teuthology.orchestra.run.smithi094.stdout:3865 punch from 0x786c497 to 0x7878c9a, (0xc803 bytes) 2024-09-21T22:51:12.643 INFO:teuthology.orchestra.run.smithi094.stdout:3866 read 0xb6ebaa5 thru 0xb6f9d92 (0xe2ee bytes) 2024-09-21T22:51:12.644 INFO:teuthology.orchestra.run.smithi094.stdout:3867 read 0x8ca692d thru 0x8cb16fc (0xadd0 bytes) 2024-09-21T22:51:12.644 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:12.644 INFO:teuthology.orchestra.run.smithi094.stdout:3871 write 0x730f2ef thru 0x731c8d2 (0xd5e4 bytes) 2024-09-21T22:51:12.657 INFO:teuthology.orchestra.run.smithi094.stdout:3872 read 0x8df76c4 thru 0x8dfdaa0 (0x63dd bytes) 2024-09-21T22:51:12.657 INFO:teuthology.orchestra.run.smithi094.stdout:3873 write 0xadc28f7 thru 0xadceae5 (0xc1ef bytes) 2024-09-21T22:51:12.670 INFO:teuthology.orchestra.run.smithi094.stdout:3874 writesame 0xc1b6711 thru 0xc1bb536 data_size 0x7(0x4e26 bytes) 2024-09-21T22:51:12.675 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:12.672+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:12.684 INFO:teuthology.orchestra.run.smithi094.stdout:3875 write 0xaec494 thru 0xaedf3f (0x1aac bytes) 2024-09-21T22:51:12.712 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.713 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.718 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:12.718 INFO:teuthology.orchestra.run.smithi094.stdout:3877 read 0x7455631 thru 0x745a5aa (0x4f7a bytes) 2024-09-21T22:51:12.718 INFO:teuthology.orchestra.run.smithi094.stdout:3878 read 0x179901b thru 0x17a601b (0xd001 bytes) 2024-09-21T22:51:12.722 INFO:teuthology.orchestra.run.smithi094.stdout:3879 clone 25 order 20 su 8192 sc 2 2024-09-21T22:51:12.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:12.817 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:12.816+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:12.877 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:12.876+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:12.883 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:12.883 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:12.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:12.978 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:12.976+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:13.060 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:13.061 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:13.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:13.078 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:13.072+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:13.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:13.179 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:13.176+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:13.192 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:13.192 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:13.208 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:13.242 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T22:51:13.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T22:51:13.312 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:13.308+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:13.319 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:13.320 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:13.339 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone24 from 0xd791f3d (overlap 0x29ffdc7) to 0xc998ee 2024-09-21T22:51:13.358 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:13.358 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:13.380 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:13.376+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:13.382 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:13.382 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:13.403 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:13.403 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:13.416 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:13.417 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:13.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:13.437 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:13.458 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:13.458 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:13.481 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:13.482 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:13.482 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:13.480+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:13.582 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:13.576+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:13.596 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:13.596 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:13.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:13.706 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:13.704+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:13.771 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:13.771 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:13.771 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:13.771 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:13.771 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:13.784 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:13.780+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:13.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:13.884 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:13.880+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:13.986 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:13.980+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:14.030 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.006319541000003, 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.002081971999999, 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.001887074000003, 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68484 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 356352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3873403369999995, 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033448665000000002, 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68484 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638316+0000", 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0100000000000001e-06 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638322+0000", 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169999999999997e-06 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638342+0000", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.995e-05 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638370+0000", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7985000000000001e-05 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671391+0000", 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033021259999999997 2024-09-21T22:51:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671483+0000", 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1380999999999998e-05 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671763+0000", 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028066199999999999 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68541 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3872646290000006, 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033895258999999997, 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68541 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638391+0000", 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638398+0000", 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9770000000000003e-06 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638421+0000", 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2578999999999998e-05 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638436+0000", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4620000000000001e-05 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671874+0000", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033438273999999997 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671995+0000", 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120567 2024-09-21T22:51:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672286+0000", 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029103199999999999 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68542 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 49152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3872000310000008, 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034317184000000001, 2024-09-21T22:51:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68542 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638456+0000", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.567e-06 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638462+0000", 2024-09-21T22:51:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9830000000000001e-06 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638487+0000", 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4199999999999999e-05 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638501+0000", 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4489e-05 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672333+0000", 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033832167000000003 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672439+0000", 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105635 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672772+0000", 2024-09-21T22:51:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033314299999999999 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68543 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3871347420000006, 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034634933999999999, 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68543 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638521+0000", 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1060000000000001e-06 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638527+0000", 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638546+0000", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8635999999999999e-05 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638559+0000", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2733e-05 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672814+0000", 2024-09-21T22:51:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034255173 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672909+0000", 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4581999999999997e-05 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673155+0000", 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024655400000000001 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68544 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 163840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3870774069999996, 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034985167999999997, 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68544 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638579+0000", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006e-06 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638584+0000", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6840000000000001e-06 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638604+0000", 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9859000000000001e-05 2024-09-21T22:51:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638618+0000", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.428e-05 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673193+0000", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034575074999999997 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673252+0000", 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8288000000000002e-05 2024-09-21T22:51:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673563+0000", 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031097600000000003 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68545 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 196608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3869385380000008, 2024-09-21T22:51:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035408569000000001, 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68545 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638718+0000", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638725+0000", 2024-09-21T22:51:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8299999999999998e-06 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638746+0000", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1566e-05 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638760+0000", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673630+0000", 2024-09-21T22:51:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034869372000000003 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673803+0000", 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000173435 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674125+0000", 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032173299999999999 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68546 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 245760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3868624529999991, 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035763809000000001, 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68546 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638794+0000", 2024-09-21T22:51:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4529999999999999e-06 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638815+0000", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1418999999999999e-05 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638837+0000", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1855999999999999e-05 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638851+0000", 2024-09-21T22:51:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.341e-05 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674168+0000", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035317861999999998 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674234+0000", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5946999999999997e-05 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674556+0000", 2024-09-21T22:51:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032186200000000002 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68547 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 278528~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3867819160000003, 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03614469, 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68547 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638874+0000", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638881+0000", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9999999999999999e-06 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638901+0000", 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999999e-05 2024-09-21T22:51:14.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638915+0000", 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4532000000000001e-05 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674611+0000", 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035696033000000002 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674712+0000", 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010040299999999999 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675018+0000", 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030589199999999999 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68548 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3867211909999995, 2024-09-21T22:51:14.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036468652999999997, 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68548 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638935+0000", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638941+0000", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638959+0000", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8431999999999999e-05 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638973+0000", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3813e-05 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675083+0000", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036110541000000003 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675199+0000", 2024-09-21T22:51:14.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115926 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675402+0000", 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020310400000000001 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68605 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:14.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3866640130000008, 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036815352000000003, 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68605 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638992+0000", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638998+0000", 2024-09-21T22:51:14.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4370000000000004e-06 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639017+0000", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639055+0000", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7441999999999998e-05 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675453+0000", 2024-09-21T22:51:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036398463999999998 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675531+0000", 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7575000000000002e-05 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675806+0000", 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027533800000000003 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68606 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.386579308, 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036991366999999997, 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68606 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.095 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639077+0000", 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:51:14.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639085+0000", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7559999999999999e-06 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639118+0000", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3705999999999999e-05 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639148+0000", 2024-09-21T22:51:14.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0162999999999999e-05 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675849+0000", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036700993000000001 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675907+0000", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8032999999999998e-05 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676067+0000", 2024-09-21T22:51:14.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015953300000000001 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68607 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 86016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3864852909999996, 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037187378, 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68607 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639171+0000", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639178+0000", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6699999999999997e-06 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639198+0000", 2024-09-21T22:51:14.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0132e-05 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639212+0000", 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4294000000000001e-05 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676124+0000", 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036912568 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676193+0000", 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8821999999999999e-05 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676357+0000", 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016368200000000001 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68608 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.103 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3864212800000004, 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037593025000000002, 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68608 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639235+0000", 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.034e-06 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639241+0000", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5830000000000003e-06 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639268+0000", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6741999999999999e-05 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639281+0000", 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.344e-05 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676415+0000", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037133806999999998 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676541+0000", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012614500000000001 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676827+0000", 2024-09-21T22:51:14.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028527400000000002 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68609 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3863540650000008, 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037858175000000001, 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.108 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68609 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:14.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639302+0000", 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0669999999999999e-06 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639309+0000", 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7429999999999996e-06 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639327+0000", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8573e-05 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639341+0000", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3286e-05 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676887+0000", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037546698000000003 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676966+0000", 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8611999999999999e-05 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677159+0000", 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000193196 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68610 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:14.112 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.386295616, 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038202400999999997, 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68610 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639360+0000", 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639367+0000", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1759999999999998e-06 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639385+0000", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8454e-05 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639399+0000", 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3835999999999999e-05 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677208+0000", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037809295999999999 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677317+0000", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108682 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677562+0000", 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024489100000000001 2024-09-21T22:51:14.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68611 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3862371479999993, 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038561235999999999, 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68611 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639419+0000", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639425+0000", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639443+0000", 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8675999999999999e-05 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639457+0000", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4083000000000001e-05 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677605+0000", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038147542999999999 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677713+0000", 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108529 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677979+0000", 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026561500000000002 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68612 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.386148918, 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038912731999999998, 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:14.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68612 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:14.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639508+0000", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639514+0000", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3500000000000002e-06 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:14.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639534+0000", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0199000000000001e-05 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639549+0000", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4056000000000001e-05 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678034+0000", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038485761 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678166+0000", 2024-09-21T22:51:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131719 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678419+0000", 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025270300000000001 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:14.125 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:14.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:14.127 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:14.120+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:14.227 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:14.224+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T22:51:14.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:51:14.280 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:14.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:14.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.280 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:124102 3.17 3:ef53ddb7:::rbd_data.111e6870f364.00000000000001df:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:14.273022+0000", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0070774590000000004, 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.007107978, 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 124102 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273022+0000", 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273022+0000", 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273025+0000", 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6180000000000002e-06 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273032+0000", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3069999999999997e-06 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273048+0000", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5917999999999999e-05 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273057+0000", 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8319999999999995e-06 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273123+0000", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6155999999999995e-05 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273222+0000", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8361000000000002e-05 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.273421+0000", 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000199571 2024-09-21T22:51:14.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274704+0000", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012829300000000001 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279839+0000", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051352730000000001 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279904+0000", 2024-09-21T22:51:14.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4301000000000007e-05 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:124106 3.1 3:831dab2c:::rbd_data.111e6870f364.00000000000001e3:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:14.274843+0000", 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0052569030000000003, 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054016309999999996, 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 124106 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274843+0000", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274843+0000", 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274845+0000", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.64e-06 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274850+0000", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7479999999999999e-06 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274864+0000", 2024-09-21T22:51:14.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3682e-05 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274873+0000", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6819999999999992e-06 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274927+0000", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4395999999999999e-05 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.274980+0000", 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3406000000000003e-05 2024-09-21T22:51:14.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.275085+0000", 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010492000000000001 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.276130+0000", 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001044756 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4382.0:124102 3.17 e74/24 ondisk, result = 0)", 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:14.279724+0000", 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00037563499999999998, 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062244500000000001, 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279724+0000", 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279724+0000", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279725+0000", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7700000000000004e-07 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279730+0000", 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5310000000000001e-06 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279764+0000", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3967000000000002e-05 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279773+0000", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3469999999999999e-06 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279825+0000", 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2629e-05 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.279833+0000", 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5889999999999997e-06 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 3 2024-09-21T22:51:14.294 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:14.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T22:51:14.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:14.345 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:14.340+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:14.429 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:14.424+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:14.467 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T22:51:14.478 INFO:teuthology.orchestra.run.smithi094.stdout:3880 write 0x6f8eb31 thru 0x6f97b9c (0x906c bytes) 2024-09-21T22:51:14.482 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:14.483 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:14.483 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:14.483 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:14.483 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:3886 read 0xae90249 thru 0xae90ff0 (0xda8 bytes) 2024-09-21T22:51:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:3891 punch from 0x5249ec4 to 0x524ff08, (0x6044 bytes) 2024-09-21T22:51:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:3894 punch from 0x9e2a777 to 0x9e2bf59, (0x17e2 bytes) 2024-09-21T22:51:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:3895 write 0xc8ef1fc thru 0xc8f46f2 (0x54f7 bytes) 2024-09-21T22:51:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:3897 write 0x4cde1a thru 0x4dac79 (0xce60 bytes) 2024-09-21T22:51:14.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:14.548 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:14.544+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:14.576 INFO:teuthology.orchestra.run.smithi094.stdout:3899 write 0xfca918 thru 0xfce362 (0x3a4b bytes) 2024-09-21T22:51:14.627 INFO:teuthology.orchestra.run.smithi094.stdout:3900 writesame 0x8086912 thru 0x808a3dc data_size 0x1d(0x3acb bytes) 2024-09-21T22:51:14.631 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:14.628+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:14.635 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.635 INFO:teuthology.orchestra.run.smithi094.stdout:3902 write 0x1d9d157 thru 0x1dab315 (0xe1bf bytes) 2024-09-21T22:51:14.680 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.680 INFO:teuthology.orchestra.run.smithi094.stdout:3904 writesame 0x19c87f1 thru 0x19d57d0 data_size 0x1(0xcfe0 bytes) 2024-09-21T22:51:14.695 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:14.695 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:14.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:14.695 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:14.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.213825723999999, 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:14.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.213786094, 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:51:14.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:51:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.213020609000001, 2024-09-21T22:51:14.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:51:14.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:51:14.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.212806660999998, 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:51:14.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:51:14.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:51:14.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.212716667999999, 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:14.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:51:14.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.212655452, 2024-09-21T22:51:14.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:14.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651814082000001, 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651800535, 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.726 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:14.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:51:14.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:51:14.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651780153000001, 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:51:14.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:51:14.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651766724000002, 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:51:14.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651752260999999, 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651739635999999, 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:14.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:51:14.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651701236000001, 2024-09-21T22:51:14.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:51:14.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651676764000001, 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:51:14.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651657899, 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:51:14.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:51:14.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:51:14.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651644661999999, 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651631688999998, 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651618894999999, 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651606394000002, 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:51:14.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:51:14.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:51:14.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:51:14.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.651295967999999, 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:14.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:14.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:14.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:14.783 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:14.784 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:3905 punch from 0x26fe748 to 0x270bdca, (0xd682 bytes) 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:3906 write 0x992a957 thru 0x9930331 (0x59db bytes) 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:3907 punch from 0x2b2a434 to 0x2b2e449, (0x4015 bytes) 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:3911 punch from 0xb77530d to 0xb77a4c2, (0x51b5 bytes) 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.785 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.786 INFO:teuthology.orchestra.run.smithi094.stdout:3916 read 0xb257f69 thru 0xb2669e0 (0xea78 bytes) 2024-09-21T22:51:14.786 INFO:teuthology.orchestra.run.smithi094.stdout:3917 write 0xe00bf49 thru 0xe014054 (0x810c bytes) 2024-09-21T22:51:14.786 INFO:teuthology.orchestra.run.smithi094.stdout:3922 write 0x93ce7b4 thru 0x93d727b (0x8ac8 bytes) 2024-09-21T22:51:14.786 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.786 INFO:teuthology.orchestra.run.smithi094.stdout:3924 writesame 0x8f03f8c thru 0x8f0be52 data_size 0x1(0x7ec7 bytes) 2024-09-21T22:51:14.786 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:14.780+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:14.788 INFO:teuthology.orchestra.run.smithi094.stdout:3925 writesame 0x729be57 thru 0x72a0279 data_size 0x1(0x4423 bytes) 2024-09-21T22:51:14.849 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.849 INFO:teuthology.orchestra.run.smithi094.stdout:3927 punch from 0x4a99818 to 0x4aa36f9, (0x9ee1 bytes) 2024-09-21T22:51:14.849 INFO:teuthology.orchestra.run.smithi094.stdout:3928 write 0xc695b63 thru 0xc698999 (0x2e37 bytes) 2024-09-21T22:51:14.857 INFO:teuthology.orchestra.run.smithi094.stdout:3930 writesame 0x2b0c030 thru 0x2b165bf data_size 0x10(0xa590 bytes) 2024-09-21T22:51:14.866 INFO:teuthology.orchestra.run.smithi094.stdout:3931 write 0x2673ebe thru 0x2682695 (0xe7d8 bytes) 2024-09-21T22:51:14.884 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:14.880+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:14.918 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:14.918 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:14.918 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:14.918 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:14.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:14.953 INFO:teuthology.orchestra.run.smithi094.stdout:3932 write 0xe2d0b0a thru 0xe2da45a (0x9951 bytes) 2024-09-21T22:51:14.959 INFO:teuthology.orchestra.run.smithi094.stdout:3933 write 0x7ad72d6 thru 0x7ae22d1 (0xaffc bytes) 2024-09-21T22:51:14.967 INFO:teuthology.orchestra.run.smithi094.stdout:3934 read 0x2b0aa20 thru 0x2b0f948 (0x4f29 bytes) 2024-09-21T22:51:14.967 INFO:teuthology.orchestra.run.smithi094.stdout:3936 read 0xd44e94e thru 0xd4514fc (0x2baf bytes) 2024-09-21T22:51:14.968 INFO:teuthology.orchestra.run.smithi094.stdout:3937 read 0x7f66ac0 thru 0x7f7198a (0xaecb bytes) 2024-09-21T22:51:14.968 INFO:teuthology.orchestra.run.smithi094.stdout:3938 read 0x9c2a99d thru 0x9c33ce5 (0x9349 bytes) 2024-09-21T22:51:14.968 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:14.974 INFO:teuthology.orchestra.run.smithi094.stdout:3940 trunc from 0xe2da45b to 0x8955180 2024-09-21T22:51:14.984 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:14.980+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:15.000 INFO:teuthology.orchestra.run.smithi094.stdout:3943 read 0x64f4f7f thru 0x6501f13 (0xcf95 bytes) 2024-09-21T22:51:15.000 INFO:teuthology.orchestra.run.smithi094.stdout:3944 read 0x7c86543 thru 0x7c9050b (0x9fc9 bytes) 2024-09-21T22:51:15.000 INFO:teuthology.orchestra.run.smithi094.stdout:3945 write 0x84b8ad4 thru 0x84c6bfd (0xe12a bytes) 2024-09-21T22:51:15.024 INFO:teuthology.orchestra.run.smithi094.stdout:3946 write 0xd043986 thru 0xd050331 (0xc9ac bytes) 2024-09-21T22:51:15.031 INFO:teuthology.orchestra.run.smithi094.stdout:3947 trunc from 0xd050332 to 0xbb101e3 2024-09-21T22:51:15.047 INFO:teuthology.orchestra.run.smithi094.stdout:3948 punch from 0x6acc634 to 0x6ad7fd6, (0xb9a2 bytes) 2024-09-21T22:51:15.047 INFO:teuthology.orchestra.run.smithi094.stdout:3950 read 0x7783d0d thru 0x778b5e5 (0x78d9 bytes) 2024-09-21T22:51:15.048 INFO:teuthology.orchestra.run.smithi094.stdout:3951 read 0xa84718b thru 0xa84f95d (0x87d3 bytes) 2024-09-21T22:51:15.051 INFO:teuthology.orchestra.run.smithi094.stdout:3952 trunc from 0xbb101e3 to 0x9b4de25 2024-09-21T22:51:15.060 INFO:teuthology.orchestra.run.smithi094.stdout:3953 punch from 0x8e7751 to 0x8ecd9a, (0x5649 bytes) 2024-09-21T22:51:15.060 INFO:teuthology.orchestra.run.smithi094.stdout:3954 read 0xaee858 thru 0xafe519 (0xfcc2 bytes) 2024-09-21T22:51:15.061 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:15.061 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:15.061 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:15.061 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:15.061 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:15.068 INFO:teuthology.orchestra.run.smithi094.stdout:3956 writesame 0x30652a5 thru 0x306f614 data_size 0x28(0xa370 bytes) 2024-09-21T22:51:15.074 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:15.078 INFO:teuthology.orchestra.run.smithi094.stdout:3957 read 0x2b0ecb4 thru 0x2b1c3e0 (0xd72d bytes) 2024-09-21T22:51:15.078 INFO:teuthology.orchestra.run.smithi094.stdout:3959 write 0x91f962d thru 0x9201e6f (0x8843 bytes) 2024-09-21T22:51:15.085 INFO:teuthology.orchestra.run.smithi094.stdout:3962 write 0x842248e thru 0x8427ff9 (0x5b6c bytes) 2024-09-21T22:51:15.086 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:15.084+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:15.091 INFO:teuthology.orchestra.run.smithi094.stdout:3963 read 0x69e0197 thru 0x69e0c4e (0xab8 bytes) 2024-09-21T22:51:15.091 INFO:teuthology.orchestra.run.smithi094.stdout:3964 read 0x7dc00d7 thru 0x7dc93cf (0x92f9 bytes) 2024-09-21T22:51:15.105 INFO:teuthology.orchestra.run.smithi094.stdout:3967 write 0xbb76b3c thru 0xbb780b5 (0x157a bytes) 2024-09-21T22:51:15.111 INFO:teuthology.orchestra.run.smithi094.stdout:3969 writesame 0x468ae47 thru 0x469168a data_size 0x2(0x6844 bytes) 2024-09-21T22:51:15.126 INFO:teuthology.orchestra.run.smithi094.stdout:3970 trunc from 0xbb780b6 to 0x7b66a84 2024-09-21T22:51:15.154 INFO:teuthology.orchestra.run.smithi094.stdout:3972 read 0xb00841 thru 0xb09a77 (0x9237 bytes) 2024-09-21T22:51:15.164 INFO:teuthology.orchestra.run.smithi094.stdout:3974 writesame 0x6cc1dc thru 0x6d9818 data_size 0x7(0xd63d bytes) 2024-09-21T22:51:15.187 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:15.184+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 30.114222108, 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:15.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34471 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 30.109047275999998, 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.039490556000000003, 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34471 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093355+0000", 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9800000000000004e-07 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093357+0000", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5210000000000001e-06 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093365+0000", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093370+0000", 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1889999999999998e-06 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128443+0000", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035073159999999999 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128522+0000", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9128000000000005e-05 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.132845+0000", 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043228559999999999 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.714022242999999, 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:15.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:15.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:15.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712507982999998, 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.217 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:15.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:15.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712454681000001, 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:15.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712399633, 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712330449, 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712267881999999, 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:15.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:15.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.712218488000001, 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.711714704999999, 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.711647125999999, 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:15.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:15.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.711097883000001, 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:15.251 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.711020401999999, 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710856271000001, 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:15.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710795292, 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:15.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:15.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710751680000001, 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:15.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:15.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:15.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.272 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710707198000001, 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710662030000002, 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:15.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:15.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.710619733000001, 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:15.285 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 25.225187144, 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:15.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:15.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:15.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:15.291 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:15.293 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:15.293 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:51:15.293 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T22:51:15.294 INFO:teuthology.orchestra.run.smithi094.stdout:3976 writesame 0x1d08086 thru 0x1d160bc data_size 0x39(0xe037 bytes) 2024-09-21T22:51:15.294 INFO:teuthology.orchestra.run.smithi094.stdout:3977 read 0x2818142 thru 0x2826a43 (0xe902 bytes) 2024-09-21T22:51:15.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T22:51:15.344 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:15.288+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:15.345 INFO:teuthology.orchestra.run.smithi094.stdout:3978 write 0x1d73a32 thru 0x1d83793 (0xfd62 bytes) 2024-09-21T22:51:15.349 INFO:teuthology.orchestra.run.smithi094.stdout:3979 write 0x483bd16 thru 0x4848cd8 (0xcfc3 bytes) 2024-09-21T22:51:15.357 INFO:teuthology.orchestra.run.smithi094.stdout:3980 read 0x56948a4 thru 0x569a926 (0x6083 bytes) 2024-09-21T22:51:15.357 INFO:teuthology.orchestra.run.smithi094.stdout:3981 write 0x1c60657 thru 0x1c65b6a (0x5514 bytes) 2024-09-21T22:51:15.360 INFO:teuthology.orchestra.run.smithi094.stdout:3982 punch from 0xffb333 to 0x100653f, (0xb20c bytes) 2024-09-21T22:51:15.361 INFO:teuthology.orchestra.run.smithi094.stdout:3985 read 0x1c11fa7 thru 0x1c1b860 (0x98ba bytes) 2024-09-21T22:51:15.361 INFO:teuthology.orchestra.run.smithi094.stdout:3986 read 0x15113e3 thru 0x151ffe2 (0xec00 bytes) 2024-09-21T22:51:15.388 INFO:teuthology.orchestra.run.smithi094.stdout:3988 writesame 0xed4b5fe thru 0xed4ffdf data_size 0x31(0x49e2 bytes) 2024-09-21T22:51:15.388 INFO:teuthology.orchestra.run.smithi094.stdout:3990 read 0xdc2c7d6 thru 0xdc3a257 (0xda82 bytes) 2024-09-21T22:51:15.388 INFO:teuthology.orchestra.run.smithi094.stdout:3991 write 0x2f05d55 thru 0x2f0e3ac (0x8658 bytes) 2024-09-21T22:51:15.393 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:15.392+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:15.396 INFO:teuthology.orchestra.run.smithi094.stdout:3992 punch from 0xc70fc4b to 0xc71510c, (0x54c1 bytes) 2024-09-21T22:51:15.396 INFO:teuthology.orchestra.run.smithi094.stdout:3994 write 0xc55f675 thru 0xc568ab1 (0x943d bytes) 2024-09-21T22:51:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:3995 trunc from 0xed4ffe0 to 0x35087f7 2024-09-21T22:51:15.461 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:15.461 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:15.461 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:15.461 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:15.474 INFO:teuthology.orchestra.run.smithi094.stdout:3996 write 0x8a10cde thru 0x8a13c87 (0x2faa bytes) 2024-09-21T22:51:15.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:15.480 INFO:teuthology.orchestra.run.smithi094.stdout:3997 writesame 0x21eafef thru 0x21eec9d data_size 0xd(0x3caf bytes) 2024-09-21T22:51:15.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:15.524 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:15.520+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:3998 read 0x69af042 thru 0x69b0276 (0x1235 bytes) 2024-09-21T22:51:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:3999 write 0x2ea8fe6 thru 0x2eb3c74 (0xac8f bytes) 2024-09-21T22:51:15.581 INFO:teuthology.orchestra.run.smithi094.stdout:4001 trunc from 0x8a13c88 to 0x1408b7e 2024-09-21T22:51:15.594 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:15.592+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:15.619 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:15.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:15.619 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:15.619 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:15.619 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:15.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:15.678 INFO:teuthology.orchestra.run.smithi094.stdout:4003 write 0xe8de69e thru 0xe8e5ca6 (0x7609 bytes) 2024-09-21T22:51:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:15.682 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:15.683 INFO:teuthology.orchestra.run.smithi094.stdout:4004 read 0x446f370 thru 0x447b865 (0xc4f6 bytes) 2024-09-21T22:51:15.684 INFO:teuthology.orchestra.run.smithi094.stdout:4006 punch from 0xb276a01 to 0xb283e86, (0xd485 bytes) 2024-09-21T22:51:15.695 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:15.692+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:15.699 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:15.700 INFO:teuthology.orchestra.run.smithi094.stdout:4007 trunc from 0xe8e5ca7 to 0x272c286 2024-09-21T22:51:15.719 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:15.719 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:15.728 INFO:teuthology.orchestra.run.smithi094.stdout:4009 writesame 0x6443b54 thru 0x644a838 data_size 0x1(0x6ce5 bytes) 2024-09-21T22:51:15.728 INFO:teuthology.orchestra.run.smithi094.stdout:4010 punch from 0x2ecd95f to 0x2ed70f5, (0x9796 bytes) 2024-09-21T22:51:15.728 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.728 INFO:teuthology.orchestra.run.smithi094.stdout:4013 punch from 0xb0940 to 0xb84d2, (0x7b92 bytes) 2024-09-21T22:51:15.732 INFO:teuthology.orchestra.run.smithi094.stdout:4015 trunc from 0x644a839 to 0x34c748d 2024-09-21T22:51:15.735 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:15.739 INFO:teuthology.orchestra.run.smithi094.stdout:4016 read 0xe18d89 thru 0xe21fe8 (0x9260 bytes) 2024-09-21T22:51:15.740 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.740 INFO:teuthology.orchestra.run.smithi094.stdout:4018 write 0xb289a5 thru 0xb37240 (0xe89c bytes) 2024-09-21T22:51:15.766 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:15.766 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:15.784 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:15.784 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:15.784 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:15.785 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:15.796 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:15.792+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:15.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:15.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:15.818 INFO:teuthology.orchestra.run.smithi094.stdout:4020 write 0xe569305 thru 0xe56c0c2 (0x2dbe bytes) 2024-09-21T22:51:15.823 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.823 INFO:teuthology.orchestra.run.smithi094.stdout:4022 read 0x2d3ed55 thru 0x2d4e6e2 (0xf98e bytes) 2024-09-21T22:51:15.824 INFO:teuthology.orchestra.run.smithi094.stdout:4025 trunc from 0xe56c0c3 to 0xd7a805c 2024-09-21T22:51:15.831 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:15.831 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:15.836 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.836 INFO:teuthology.orchestra.run.smithi094.stdout:4027 read 0xd52656 thru 0xd6031d (0xdcc8 bytes) 2024-09-21T22:51:15.841 INFO:teuthology.orchestra.run.smithi094.stdout:4028 read 0x2529274 thru 0x252b6d9 (0x2466 bytes) 2024-09-21T22:51:15.841 INFO:teuthology.orchestra.run.smithi094.stdout:4029 punch from 0x3f33526 to 0x3f42213, (0xeced bytes) 2024-09-21T22:51:15.841 INFO:teuthology.orchestra.run.smithi094.stdout:4030 write 0x904bfd8 thru 0x9057597 (0xb5c0 bytes) 2024-09-21T22:51:15.847 INFO:teuthology.orchestra.run.smithi094.stdout:4031 read 0x8a77916 thru 0x8a8271a (0xae05 bytes) 2024-09-21T22:51:15.847 INFO:teuthology.orchestra.run.smithi094.stdout:4032 read 0xc0b6085 thru 0xc0bc609 (0x6585 bytes) 2024-09-21T22:51:15.847 INFO:teuthology.orchestra.run.smithi094.stdout:4033 read 0x3f7e853 thru 0x3f881ee (0x999c bytes) 2024-09-21T22:51:15.847 INFO:teuthology.orchestra.run.smithi094.stdout:4034 read 0xb5ea505 thru 0xb5f321b (0x8d17 bytes) 2024-09-21T22:51:15.856 INFO:teuthology.orchestra.run.smithi094.stdout:4036 write 0xe37bf27 thru 0xe38283d (0x6917 bytes) 2024-09-21T22:51:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:4037 trunc from 0xe38283e to 0xcba07ae 2024-09-21T22:51:15.880 INFO:teuthology.orchestra.run.smithi094.stdout:4039 trunc from 0xcba07ae to 0xed214d9 2024-09-21T22:51:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4040 writesame 0x149b997 thru 0x14a4ba7 data_size 0x1(0x9211 bytes) 2024-09-21T22:51:15.896 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:15.892+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:15.898 INFO:teuthology.orchestra.run.smithi094.stdout:4041 writesame 0x2c558f4 thru 0x2c5a3b2 data_size 0x2b(0x4abf bytes) 2024-09-21T22:51:15.903 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.903 INFO:teuthology.orchestra.run.smithi094.stdout:4043 write 0x60f8a42 thru 0x6108430 (0xf9ef bytes) 2024-09-21T22:51:15.908 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:15.908 INFO:teuthology.orchestra.run.smithi094.stdout:4046 read 0x1b4f8a thru 0x1bfc91 (0xad08 bytes) 2024-09-21T22:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:4047 write 0x44415f1 thru 0x4441e86 (0x896 bytes) 2024-09-21T22:51:15.933 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:15.933 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:15.936 INFO:teuthology.orchestra.run.smithi094.stdout:4048 trunc from 0xed214d9 to 0x2de1cac 2024-09-21T22:51:15.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:15.973 INFO:teuthology.orchestra.run.smithi094.stdout:4049 write 0x656ebd8 thru 0x657794c (0x8d75 bytes) 2024-09-21T22:51:15.981 INFO:teuthology.orchestra.run.smithi094.stdout:4050 punch from 0x495fd82 to 0x49615b9, (0x1837 bytes) 2024-09-21T22:51:15.981 INFO:teuthology.orchestra.run.smithi094.stdout:4053 punch from 0xaf67d1 to 0xb027a7, (0xbfd6 bytes) 2024-09-21T22:51:15.987 INFO:teuthology.orchestra.run.smithi094.stdout:4054 trunc from 0x657794d to 0x11487ba 2024-09-21T22:51:15.998 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:15.996+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:16.098 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:16.096+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:16.108 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:16.109 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:16.123 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:16.163 INFO:teuthology.orchestra.run.smithi094.stdout:4055 write 0x731763f thru 0x731ee07 (0x77c9 bytes) 2024-09-21T22:51:16.169 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.178 INFO:teuthology.orchestra.run.smithi094.stdout:4057 writesame 0x7e0034d thru 0x7e0abd9 data_size 0x13(0xa88d bytes) 2024-09-21T22:51:16.179 INFO:teuthology.orchestra.run.smithi094.stdout:4058 write 0x22615fb thru 0x226647e (0x4e84 bytes) 2024-09-21T22:51:16.185 INFO:teuthology.orchestra.run.smithi094.stdout:4059 read 0x2c0ac88 thru 0x2c17425 (0xc79e bytes) 2024-09-21T22:51:16.185 INFO:teuthology.orchestra.run.smithi094.stdout:4060 writesame 0x44eee2a thru 0x44f2052 data_size 0x1(0x3229 bytes) 2024-09-21T22:51:16.198 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:16.196+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:4061 trunc from 0x7e0abda to 0xa7b4893 2024-09-21T22:51:16.209 INFO:teuthology.orchestra.run.smithi094.stdout:4062 write 0xa64bc85 thru 0xa650291 (0x460d bytes) 2024-09-21T22:51:16.215 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.215 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.215 INFO:teuthology.orchestra.run.smithi094.stdout:4066 writesame 0xa23768b thru 0xa23982b data_size 0x1(0x21a1 bytes) 2024-09-21T22:51:16.238 INFO:teuthology.orchestra.run.smithi094.stdout:4067 writesame 0xdbaf6a4 thru 0xdbbaa93 data_size 0x8(0xb3f0 bytes) 2024-09-21T22:51:16.238 INFO:teuthology.orchestra.run.smithi094.stdout:4068 write 0x108d9a6 thru 0x108efc2 (0x161d bytes) 2024-09-21T22:51:16.240 INFO:teuthology.orchestra.run.smithi094.stdout:4069 punch from 0xb2038cd to 0xb209fb9, (0x66ec bytes) 2024-09-21T22:51:16.254 INFO:teuthology.orchestra.run.smithi094.stdout:4070 trunc from 0xdbbaa94 to 0x46c6130 2024-09-21T22:51:16.270 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T22:51:16.285 INFO:teuthology.orchestra.run.smithi094.stdout:4071 write 0x3a3bf31 thru 0x3a44574 (0x8644 bytes) 2024-09-21T22:51:16.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T22:51:16.299 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:16.296+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:16.315 INFO:teuthology.orchestra.run.smithi094.stdout:4072 write 0xb38dcee thru 0xb38fb89 (0x1e9c bytes) 2024-09-21T22:51:16.317 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:16.317 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:16.322 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:16.337 INFO:teuthology.orchestra.run.smithi094.stdout:4075 writesame 0xcbcfa71 thru 0xcbdad2b data_size 0x1(0xb2bb bytes) 2024-09-21T22:51:16.337 INFO:teuthology.orchestra.run.smithi094.stdout:4076 write 0xadd21c thru 0xaeb279 (0xe05e bytes) 2024-09-21T22:51:16.344 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.344 INFO:teuthology.orchestra.run.smithi094.stdout:4078 write 0x4a886d9 thru 0x4a954b6 (0xcdde bytes) 2024-09-21T22:51:16.349 INFO:teuthology.orchestra.run.smithi094.stdout:4081 write 0xab2b8ee thru 0xab36df1 (0xb504 bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4082 read 0x2a1791c thru 0x2a1f41a (0x7aff bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4083 read 0x7056193 thru 0x705fb64 (0x99d2 bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4084 punch from 0x6186c76 to 0x618a7f4, (0x3b7e bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4086 read 0x16c6a7b thru 0x16cfcc8 (0x924e bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4087 read 0x5f27ce4 thru 0x5f2ef89 (0x72a6 bytes) 2024-09-21T22:51:16.355 INFO:teuthology.orchestra.run.smithi094.stdout:4089 punch from 0x6f643b to 0x70050c, (0xa0d1 bytes) 2024-09-21T22:51:16.356 INFO:teuthology.orchestra.run.smithi094.stdout:4090 trunc from 0xcbdad2c to 0xe0ee053 2024-09-21T22:51:16.364 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.364 INFO:teuthology.orchestra.run.smithi094.stdout:4094 writesame 0x85036d3 thru 0x85046ea data_size 0x14(0x1018 bytes) 2024-09-21T22:51:16.368 INFO:teuthology.orchestra.run.smithi094.stdout:4095 write 0xc98002 thru 0xca5003 (0xd002 bytes) 2024-09-21T22:51:16.400 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:16.396+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:16.430 INFO:teuthology.orchestra.run.smithi094.stdout:4096 writesame 0x77c56c thru 0x77f01f data_size 0x6(0x2ab4 bytes) 2024-09-21T22:51:16.435 INFO:teuthology.orchestra.run.smithi094.stdout:4097 writesame 0x14d8eb4 thru 0x14db6db data_size 0x14(0x2828 bytes) 2024-09-21T22:51:16.441 INFO:teuthology.orchestra.run.smithi094.stdout:4099 punch from 0x8028054 to 0x80358d2, (0xd87e bytes) 2024-09-21T22:51:16.441 INFO:teuthology.orchestra.run.smithi094.stdout:4100 write 0xa82dde3 thru 0xa8301c6 (0x23e4 bytes) 2024-09-21T22:51:16.446 INFO:teuthology.orchestra.run.smithi094.stdout:4101 writesame 0xca94a03 thru 0xca9f712 data_size 0x4e(0xad10 bytes) 2024-09-21T22:51:16.451 INFO:teuthology.orchestra.run.smithi094.stdout:4102 write 0x59f4457 thru 0x59f8188 (0x3d32 bytes) 2024-09-21T22:51:16.456 INFO:teuthology.orchestra.run.smithi094.stdout:4104 read 0x4615d96 thru 0x461ed0f (0x8f7a bytes) 2024-09-21T22:51:16.475 INFO:teuthology.orchestra.run.smithi094.stdout:4106 trunc from 0xe0ee053 to 0xd0e4f2 2024-09-21T22:51:16.489 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:16.489 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:16.498 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.498 INFO:teuthology.orchestra.run.smithi094.stdout:4109 read 0x8ef4ca thru 0x8f163e (0x2175 bytes) 2024-09-21T22:51:16.500 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:16.496+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:16.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:16.515 INFO:teuthology.orchestra.run.smithi094.stdout:4110 write 0x6778296 thru 0x678717a (0xeee5 bytes) 2024-09-21T22:51:16.520 INFO:teuthology.orchestra.run.smithi094.stdout:4111 read 0x7f2f83 thru 0x7f68fe (0x397c bytes) 2024-09-21T22:51:16.532 INFO:teuthology.orchestra.run.smithi094.stdout:4112 write 0x8b7e518 thru 0x8b882ab (0x9d94 bytes) 2024-09-21T22:51:16.537 INFO:teuthology.orchestra.run.smithi094.stdout:4113 read 0x23bf2f2 thru 0x23c1284 (0x1f93 bytes) 2024-09-21T22:51:16.537 INFO:teuthology.orchestra.run.smithi094.stdout:4114 read 0xe2310 thru 0xe823c (0x5f2d bytes) 2024-09-21T22:51:16.541 INFO:teuthology.orchestra.run.smithi094.stdout:4115 read 0x4ece08e thru 0x4edc60c (0xe57f bytes) 2024-09-21T22:51:16.542 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:16.551 INFO:teuthology.orchestra.run.smithi094.stdout:4117 trunc from 0x8b882ac to 0x1d277ec 2024-09-21T22:51:16.570 INFO:teuthology.orchestra.run.smithi094.stdout:4118 trunc from 0x1d277ec to 0x4680b9c 2024-09-21T22:51:16.576 INFO:teuthology.orchestra.run.smithi094.stdout:4119 punch from 0x15fcac6 to 0x160a5ea, (0xdb24 bytes) 2024-09-21T22:51:16.598 INFO:teuthology.orchestra.run.smithi094.stdout:4120 write 0xe3dd48a thru 0xe3e29ab (0x5522 bytes) 2024-09-21T22:51:16.600 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:16.596+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:16.604 INFO:teuthology.orchestra.run.smithi094.stdout:4121 clone 26 order 20 su 65536 sc 9 2024-09-21T22:51:16.672 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:16.672 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:16.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:16.701 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:16.696+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:16.802 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:16.800+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:16.854 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:16.854 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:16.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:16.910 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:16.908+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:17.004 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:17.000+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:17.032 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.032 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:17.105 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:17.100+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:17.206 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:17.200+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:17.212 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.212 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T22:51:17.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:51:17.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T22:51:17.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:51:17.311 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:17.304+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:17.352 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.353 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:17.395 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone25 from 0xe3e29ac (overlap 0xd0e4f2) to 0x0 2024-09-21T22:51:17.414 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:17.412+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:17.507 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:17.500+0000 7f8b56625640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26523) UID: 0 2024-09-21T22:51:17.547 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.548 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:17.653 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:17.651+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:17.705 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:51:17.708 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:17.703+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:17.722 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-21T22:51:17.291193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514411520,"num_objects":2168,"num_object_clones":619,"num_object_copies":4336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2168,"num_whiteouts":1046,"num_read":159121,"num_read_kb":376647,"num_write":17443,"num_write_kb":174520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":3513975,"num_keys_recovered":33,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14201,"ondisk_log_size":14201,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":521092,"kb_used_data":207924,"kb_used_omap":15,"kb_used_meta":312880,"kb_avail":754453628,"statfs":{"total":773094113280,"available":772560515072,"internally_reserved":0,"allocated":212914176,"data_stored":245060470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15556,"internal_metadata":320389948},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42516623,"num_objects":260,"num_object_clones":61,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":139,"num_read":26621,"num_read_kb":35102,"num_write":1242,"num_write_kb":9943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000672"},"pg_stats":[{"pgid":"3.1f","version":"73'855","reported_seq":6706,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.311295+0000","last_change":"2024-09-21T22:50:57.035011+0000","last_active":"2024-09-21T22:51:13.311295+0000","last_peered":"2024-09-21T22:51:13.311295+0000","last_clean":"2024-09-21T22:51:13.311295+0000","last_became_active":"2024-09-21T22:50:57.033659+0000","last_became_peered":"2024-09-21T22:50:57.033659+0000","last_unstale":"2024-09-21T22:51:13.311295+0000","last_undegraded":"2024-09-21T22:51:13.311295+0000","last_fullsized":"2024-09-21T22:51:13.311295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:02.219936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730179999999999,"stat_sum":{"num_bytes":30508004,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":32,"num_read":9055,"num_read_kb":19733,"num_write":1137,"num_write_kb":6827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":3100812,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'381","reported_seq":5564,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.454035+0000","last_change":"2024-09-21T22:50:37.035523+0000","last_active":"2024-09-21T22:51:13.454035+0000","last_peered":"2024-09-21T22:51:13.454035+0000","last_clean":"2024-09-21T22:51:13.454035+0000","last_became_active":"2024-09-21T22:50:37.030715+0000","last_became_peered":"2024-09-21T22:50:37.030715+0000","last_unstale":"2024-09-21T22:51:13.454035+0000","last_undegraded":"2024-09-21T22:51:13.454035+0000","last_fullsized":"2024-09-21T22:51:13.454035+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:02:32.907226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020037760000000001,"stat_sum":{"num_bytes":24400264,"num_objects":69,"num_object_clones":16,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":6482,"num_read_kb":18846,"num_write":349,"num_write_kb":2558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":85270,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"76'913","reported_seq":6617,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.364406+0000","last_change":"2024-09-21T22:50:09.045034+0000","last_active":"2024-09-21T22:51:16.364406+0000","last_peered":"2024-09-21T22:51:16.364406+0000","last_clean":"2024-09-21T22:51:16.364406+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:51:16.364406+0000","last_undegraded":"2024-09-21T22:51:16.364406+0000","last_fullsized":"2024-09-21T22:51:16.364406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:55.566144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010806819999999999,"stat_sum":{"num_bytes":26385297,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":9633,"num_read_kb":41238,"num_write":1399,"num_write_kb":14767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"76'308","reported_seq":1285,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279400+0000","last_change":"2024-09-21T22:50:09.046692+0000","last_active":"2024-09-21T22:51:16.279400+0000","last_peered":"2024-09-21T22:51:16.279400+0000","last_clean":"2024-09-21T22:51:16.279400+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:51:16.279400+0000","last_undegraded":"2024-09-21T22:51:16.279400+0000","last_fullsized":"2024-09-21T22:51:16.279400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:48.393370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002495611,"stat_sum":{"num_bytes":6562536,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":36,"num_read":1361,"num_read_kb":7259,"num_write":304,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"74'900","reported_seq":5062,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273989+0000","last_change":"2024-09-21T22:50:09.054590+0000","last_active":"2024-09-21T22:51:15.273989+0000","last_peered":"2024-09-21T22:51:15.273989+0000","last_clean":"2024-09-21T22:51:15.273989+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:51:15.273989+0000","last_undegraded":"2024-09-21T22:51:15.273989+0000","last_fullsized":"2024-09-21T22:51:15.273989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:10.529574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001622592,"stat_sum":{"num_bytes":7344583,"num_objects":63,"num_object_clones":22,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":7909,"num_read_kb":11374,"num_write":1435,"num_write_kb":4853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'657","reported_seq":6058,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.454418+0000","last_change":"2024-09-21T22:50:09.031950+0000","last_active":"2024-09-21T22:51:15.454418+0000","last_peered":"2024-09-21T22:51:15.454418+0000","last_clean":"2024-09-21T22:51:15.454418+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:51:15.454418+0000","last_undegraded":"2024-09-21T22:51:15.454418+0000","last_fullsized":"2024-09-21T22:51:15.454418+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:03:39.571273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002297323,"stat_sum":{"num_bytes":9569854,"num_objects":69,"num_object_clones":24,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":7502,"num_read_kb":8004,"num_write":978,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"75'376","reported_seq":4010,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266293+0000","last_change":"2024-09-21T22:50:09.032544+0000","last_active":"2024-09-21T22:51:15.266293+0000","last_peered":"2024-09-21T22:51:15.266293+0000","last_clean":"2024-09-21T22:51:15.266293+0000","last_became_active":"2024-09-21T22:50:07.033757+0000","last_became_peered":"2024-09-21T22:50:07.033757+0000","last_unstale":"2024-09-21T22:51:15.266293+0000","last_undegraded":"2024-09-21T22:51:15.266293+0000","last_fullsized":"2024-09-21T22:51:15.266293+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:38:01.192114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084097000000000002,"stat_sum":{"num_bytes":12353021,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4529,"num_read_kb":12087,"num_write":395,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"73'896","reported_seq":7978,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.457816+0000","last_change":"2024-09-21T22:50:09.027841+0000","last_active":"2024-09-21T22:51:13.457816+0000","last_peered":"2024-09-21T22:51:13.457816+0000","last_clean":"2024-09-21T22:51:13.457816+0000","last_became_active":"2024-09-21T22:50:07.034008+0000","last_became_peered":"2024-09-21T22:50:07.034008+0000","last_unstale":"2024-09-21T22:51:13.457816+0000","last_undegraded":"2024-09-21T22:51:13.457816+0000","last_fullsized":"2024-09-21T22:51:13.457816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:21.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020218739999999999,"stat_sum":{"num_bytes":19733066,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":11049,"num_read_kb":20725,"num_write":1527,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"74'266","reported_seq":2126,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273845+0000","last_change":"2024-09-21T22:50:09.054285+0000","last_active":"2024-09-21T22:51:15.273845+0000","last_peered":"2024-09-21T22:51:15.273845+0000","last_clean":"2024-09-21T22:51:15.273845+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:51:15.273845+0000","last_undegraded":"2024-09-21T22:51:15.273845+0000","last_fullsized":"2024-09-21T22:51:15.273845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:12.184069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013918940000000001,"stat_sum":{"num_bytes":6160592,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":35,"num_read":1916,"num_read_kb":4004,"num_write":267,"num_write_kb":2440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"76'295","reported_seq":6706,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.424291+0000","last_change":"2024-09-21T22:50:09.045475+0000","last_active":"2024-09-21T22:51:16.424291+0000","last_peered":"2024-09-21T22:51:16.424291+0000","last_clean":"2024-09-21T22:51:16.424291+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:51:16.424291+0000","last_undegraded":"2024-09-21T22:51:16.424291+0000","last_fullsized":"2024-09-21T22:51:16.424291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:49.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504238,"stat_sum":{"num_bytes":18228670,"num_objects":80,"num_object_clones":24,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":6371,"num_read_kb":6196,"num_write":256,"num_write_kb":3667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"74'308","reported_seq":10653,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258662+0000","last_change":"2024-09-21T22:50:09.062094+0000","last_active":"2024-09-21T22:51:15.258662+0000","last_peered":"2024-09-21T22:51:15.258662+0000","last_clean":"2024-09-21T22:51:15.258662+0000","last_became_active":"2024-09-21T22:50:07.039164+0000","last_became_peered":"2024-09-21T22:50:07.039164+0000","last_unstale":"2024-09-21T22:51:15.258662+0000","last_undegraded":"2024-09-21T22:51:15.258662+0000","last_fullsized":"2024-09-21T22:51:15.258662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:44.821161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081156800000000003,"stat_sum":{"num_bytes":34643350,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":10646,"num_read_kb":37673,"num_write":301,"num_write_kb":15178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"74'387","reported_seq":6287,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.271524+0000","last_change":"2024-09-21T22:50:09.032016+0000","last_active":"2024-09-21T22:51:15.271524+0000","last_peered":"2024-09-21T22:51:15.271524+0000","last_clean":"2024-09-21T22:51:15.271524+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:51:15.271524+0000","last_undegraded":"2024-09-21T22:51:15.271524+0000","last_fullsized":"2024-09-21T22:51:15.271524+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:46.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021866500000000001,"stat_sum":{"num_bytes":22342776,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":32,"num_read":6462,"num_read_kb":9579,"num_write":462,"num_write_kb":9838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.432801+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:51:16.432801+0000","last_peered":"2024-09-21T22:51:16.432801+0000","last_clean":"2024-09-21T22:51:16.432801+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:51:16.432801+0000","last_undegraded":"2024-09-21T22:51:16.432801+0000","last_fullsized":"2024-09-21T22:51:16.432801+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"74'284","reported_seq":5965,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273848+0000","last_change":"2024-09-21T22:50:09.053734+0000","last_active":"2024-09-21T22:51:15.273848+0000","last_peered":"2024-09-21T22:51:15.273848+0000","last_clean":"2024-09-21T22:51:15.273848+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:51:15.273848+0000","last_undegraded":"2024-09-21T22:51:15.273848+0000","last_fullsized":"2024-09-21T22:51:15.273848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:29.531204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003049,"stat_sum":{"num_bytes":17136162,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":5634,"num_read_kb":13409,"num_write":281,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266348+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:51:15.266348+0000","last_peered":"2024-09-21T22:51:15.266348+0000","last_clean":"2024-09-21T22:51:15.266348+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:51:15.266348+0000","last_undegraded":"2024-09-21T22:51:15.266348+0000","last_fullsized":"2024-09-21T22:51:15.266348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"73'400","reported_seq":1861,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.457646+0000","last_change":"2024-09-21T22:50:09.027492+0000","last_active":"2024-09-21T22:51:13.457646+0000","last_peered":"2024-09-21T22:51:13.457646+0000","last_clean":"2024-09-21T22:51:13.457646+0000","last_became_active":"2024-09-21T22:50:07.028763+0000","last_became_peered":"2024-09-21T22:50:07.028763+0000","last_unstale":"2024-09-21T22:51:13.457646+0000","last_undegraded":"2024-09-21T22:51:13.457646+0000","last_fullsized":"2024-09-21T22:51:13.457646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:39.923572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018249729999999999,"stat_sum":{"num_bytes":5871212,"num_objects":67,"num_object_clones":21,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":2537,"num_read_kb":4130,"num_write":462,"num_write_kb":2736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258755+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:51:15.258755+0000","last_peered":"2024-09-21T22:51:15.258755+0000","last_clean":"2024-09-21T22:51:15.258755+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:51:15.258755+0000","last_undegraded":"2024-09-21T22:51:15.258755+0000","last_fullsized":"2024-09-21T22:51:15.258755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'489","reported_seq":2236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266378+0000","last_change":"2024-09-21T22:50:09.032823+0000","last_active":"2024-09-21T22:51:15.266378+0000","last_peered":"2024-09-21T22:51:15.266378+0000","last_clean":"2024-09-21T22:51:15.266378+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:51:15.266378+0000","last_undegraded":"2024-09-21T22:51:15.266378+0000","last_fullsized":"2024-09-21T22:51:15.266378+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:04:18.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093496699999999998,"stat_sum":{"num_bytes":8336757,"num_objects":73,"num_object_clones":24,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":2721,"num_read_kb":2833,"num_write":610,"num_write_kb":3557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266349+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:51:15.266349+0000","last_peered":"2024-09-21T22:51:15.266349+0000","last_clean":"2024-09-21T22:51:15.266349+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:51:15.266349+0000","last_undegraded":"2024-09-21T22:51:15.266349+0000","last_fullsized":"2024-09-21T22:51:15.266349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'229","reported_seq":751,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279946+0000","last_change":"2024-09-21T22:50:09.046691+0000","last_active":"2024-09-21T22:51:16.279946+0000","last_peered":"2024-09-21T22:51:16.279946+0000","last_clean":"2024-09-21T22:51:16.279946+0000","last_became_active":"2024-09-21T22:50:07.039630+0000","last_became_peered":"2024-09-21T22:50:07.039630+0000","last_unstale":"2024-09-21T22:51:16.279946+0000","last_undegraded":"2024-09-21T22:51:16.279946+0000","last_fullsized":"2024-09-21T22:51:16.279946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:22:56.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021691330000000002,"stat_sum":{"num_bytes":8327811,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":457,"num_read_kb":9110,"num_write":258,"num_write_kb":7708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'350","reported_seq":3838,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.271602+0000","last_change":"2024-09-21T22:50:09.030568+0000","last_active":"2024-09-21T22:51:15.271602+0000","last_peered":"2024-09-21T22:51:15.271602+0000","last_clean":"2024-09-21T22:51:15.271602+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:51:15.271602+0000","last_undegraded":"2024-09-21T22:51:15.271602+0000","last_fullsized":"2024-09-21T22:51:15.271602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:19.849107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079408,"stat_sum":{"num_bytes":15531119,"num_objects":63,"num_object_clones":21,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":4541,"num_read_kb":8357,"num_write":336,"num_write_kb":5019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279188+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:51:16.279188+0000","last_peered":"2024-09-21T22:51:16.279188+0000","last_clean":"2024-09-21T22:51:16.279188+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:51:16.279188+0000","last_undegraded":"2024-09-21T22:51:16.279188+0000","last_fullsized":"2024-09-21T22:51:16.279188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"75'487","reported_seq":4346,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.400774+0000","last_change":"2024-09-21T22:50:09.032584+0000","last_active":"2024-09-21T22:51:15.400774+0000","last_peered":"2024-09-21T22:51:15.400774+0000","last_clean":"2024-09-21T22:51:15.400774+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:51:15.400774+0000","last_undegraded":"2024-09-21T22:51:15.400774+0000","last_fullsized":"2024-09-21T22:51:15.400774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:05.104110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076847199999999997,"stat_sum":{"num_bytes":24377689,"num_objects":79,"num_object_clones":27,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":37,"num_read":5446,"num_read_kb":15899,"num_write":735,"num_write_kb":9286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258435+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:51:15.258435+0000","last_peered":"2024-09-21T22:51:15.258435+0000","last_clean":"2024-09-21T22:51:15.258435+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:51:15.258435+0000","last_undegraded":"2024-09-21T22:51:15.258435+0000","last_fullsized":"2024-09-21T22:51:15.258435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"75'283","reported_seq":1299,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.425459+0000","last_change":"2024-09-21T22:50:09.053334+0000","last_active":"2024-09-21T22:51:15.425459+0000","last_peered":"2024-09-21T22:51:15.425459+0000","last_clean":"2024-09-21T22:51:15.425459+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:51:15.425459+0000","last_undegraded":"2024-09-21T22:51:15.425459+0000","last_fullsized":"2024-09-21T22:51:15.425459+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:04.019239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068683399999999999,"stat_sum":{"num_bytes":15887827,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":37,"num_read":1021,"num_read_kb":4914,"num_write":247,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258852+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:51:15.258852+0000","last_peered":"2024-09-21T22:51:15.258852+0000","last_clean":"2024-09-21T22:51:15.258852+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:51:15.258852+0000","last_undegraded":"2024-09-21T22:51:15.258852+0000","last_fullsized":"2024-09-21T22:51:15.258852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"71'492","reported_seq":1975,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.252923+0000","last_change":"2024-09-21T22:50:09.026491+0000","last_active":"2024-09-21T22:51:13.252923+0000","last_peered":"2024-09-21T22:51:13.252923+0000","last_clean":"2024-09-21T22:51:13.252923+0000","last_became_active":"2024-09-21T22:50:07.028774+0000","last_became_peered":"2024-09-21T22:50:07.028774+0000","last_unstale":"2024-09-21T22:51:13.252923+0000","last_undegraded":"2024-09-21T22:51:13.252923+0000","last_fullsized":"2024-09-21T22:51:13.252923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:37:30.581308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010115009999999999,"stat_sum":{"num_bytes":11862562,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":28,"num_read":2570,"num_read_kb":6859,"num_write":657,"num_write_kb":5385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279219+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:51:16.279219+0000","last_peered":"2024-09-21T22:51:16.279219+0000","last_clean":"2024-09-21T22:51:16.279219+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:51:16.279219+0000","last_undegraded":"2024-09-21T22:51:16.279219+0000","last_fullsized":"2024-09-21T22:51:16.279219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258624+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:51:15.258624+0000","last_peered":"2024-09-21T22:51:15.258624+0000","last_clean":"2024-09-21T22:51:15.258624+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:51:15.258624+0000","last_undegraded":"2024-09-21T22:51:15.258624+0000","last_fullsized":"2024-09-21T22:51:15.258624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"73'262","reported_seq":2013,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.313501+0000","last_change":"2024-09-21T22:50:09.026886+0000","last_active":"2024-09-21T22:51:13.313501+0000","last_peered":"2024-09-21T22:51:13.313501+0000","last_clean":"2024-09-21T22:51:13.313501+0000","last_became_active":"2024-09-21T22:50:07.032070+0000","last_became_peered":"2024-09-21T22:50:07.032070+0000","last_unstale":"2024-09-21T22:51:13.313501+0000","last_undegraded":"2024-09-21T22:51:13.313501+0000","last_fullsized":"2024-09-21T22:51:13.313501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:11.597978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001477641,"stat_sum":{"num_bytes":12396492,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":2144,"num_read_kb":2316,"num_write":254,"num_write_kb":1297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"75'244","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.429986+0000","last_change":"2024-09-21T22:50:09.030512+0000","last_active":"2024-09-21T22:51:15.429986+0000","last_peered":"2024-09-21T22:51:15.429986+0000","last_clean":"2024-09-21T22:51:15.429986+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:51:15.429986+0000","last_undegraded":"2024-09-21T22:51:15.429986+0000","last_fullsized":"2024-09-21T22:51:15.429986+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:11.890240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00126947,"stat_sum":{"num_bytes":6511302,"num_objects":64,"num_object_clones":19,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":37,"num_read":3354,"num_read_kb":4212,"num_write":204,"num_write_kb":1562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"76'215","reported_seq":633,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.485129+0000","last_change":"2024-09-21T22:50:34.240149+0000","last_active":"2024-09-21T22:51:16.485129+0000","last_peered":"2024-09-21T22:51:16.485129+0000","last_clean":"2024-09-21T22:51:16.485129+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:51:16.485129+0000","last_undegraded":"2024-09-21T22:51:16.485129+0000","last_fullsized":"2024-09-21T22:51:16.485129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:29:39.305976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092677,"stat_sum":{"num_bytes":5339608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":409,"num_read_kb":1322,"num_write":202,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"72'256","reported_seq":1232,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.253039+0000","last_change":"2024-09-21T22:50:09.027738+0000","last_active":"2024-09-21T22:51:13.253039+0000","last_peered":"2024-09-21T22:51:13.253039+0000","last_clean":"2024-09-21T22:51:13.253039+0000","last_became_active":"2024-09-21T22:50:07.037670+0000","last_became_peered":"2024-09-21T22:50:07.037670+0000","last_unstale":"2024-09-21T22:51:13.253039+0000","last_undegraded":"2024-09-21T22:51:13.253039+0000","last_fullsized":"2024-09-21T22:51:13.253039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:02.862433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002153528,"stat_sum":{"num_bytes":10954015,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":30,"num_read":925,"num_read_kb":4705,"num_write":228,"num_write_kb":4583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'266","reported_seq":4981,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279296+0000","last_change":"2024-09-21T22:50:09.044903+0000","last_active":"2024-09-21T22:51:16.279296+0000","last_peered":"2024-09-21T22:51:16.279296+0000","last_clean":"2024-09-21T22:51:16.279296+0000","last_became_active":"2024-09-21T22:50:07.035007+0000","last_became_peered":"2024-09-21T22:50:07.035007+0000","last_unstale":"2024-09-21T22:51:16.279296+0000","last_undegraded":"2024-09-21T22:51:16.279296+0000","last_fullsized":"2024-09-21T22:51:16.279296+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:29:32.095871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261420000000001,"stat_sum":{"num_bytes":18079611,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":4822,"num_read_kb":13795,"num_write":285,"num_write_kb":5826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"74'435","reported_seq":2815,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258714+0000","last_change":"2024-09-21T22:50:09.064319+0000","last_active":"2024-09-21T22:51:15.258714+0000","last_peered":"2024-09-21T22:51:15.258714+0000","last_clean":"2024-09-21T22:51:15.258714+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:51:15.258714+0000","last_undegraded":"2024-09-21T22:51:15.258714+0000","last_fullsized":"2024-09-21T22:51:15.258714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:52:11.181511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024500839999999999,"stat_sum":{"num_bytes":6906500,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":4507,"num_read_kb":7629,"num_write":421,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"75'349","reported_seq":4162,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.402976+0000","last_change":"2024-09-21T22:50:09.064534+0000","last_active":"2024-09-21T22:51:15.402976+0000","last_peered":"2024-09-21T22:51:15.402976+0000","last_clean":"2024-09-21T22:51:15.402976+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:51:15.402976+0000","last_undegraded":"2024-09-21T22:51:15.402976+0000","last_fullsized":"2024-09-21T22:51:15.402976+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:53:34.466465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002506234,"stat_sum":{"num_bytes":18508847,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":4543,"num_read_kb":8378,"num_write":399,"num_write_kb":4576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"74'377","reported_seq":4863,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258901+0000","last_change":"2024-09-21T22:50:09.064820+0000","last_active":"2024-09-21T22:51:15.258901+0000","last_peered":"2024-09-21T22:51:15.258901+0000","last_clean":"2024-09-21T22:51:15.258901+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:51:15.258901+0000","last_undegraded":"2024-09-21T22:51:15.258901+0000","last_fullsized":"2024-09-21T22:51:15.258901+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:27.338219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021047100000000001,"stat_sum":{"num_bytes":27316827,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5761,"num_read_kb":8654,"num_write":418,"num_write_kb":2651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"76'438","reported_seq":6192,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.536991+0000","last_change":"2024-09-21T22:50:37.035011+0000","last_active":"2024-09-21T22:51:16.536991+0000","last_peered":"2024-09-21T22:51:16.536991+0000","last_clean":"2024-09-21T22:51:16.536991+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:51:16.536991+0000","last_undegraded":"2024-09-21T22:51:16.536991+0000","last_fullsized":"2024-09-21T22:51:16.536991+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:02.613006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001972101,"stat_sum":{"num_bytes":20647928,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":38,"num_read":7182,"num_read_kb":17418,"num_write":429,"num_write_kb":6292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"75'803","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258663+0000","last_change":"2024-09-21T22:50:09.062805+0000","last_active":"2024-09-21T22:51:15.258663+0000","last_peered":"2024-09-21T22:51:15.258663+0000","last_clean":"2024-09-21T22:51:15.258663+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:51:15.258663+0000","last_undegraded":"2024-09-21T22:51:15.258663+0000","last_fullsized":"2024-09-21T22:51:15.258663+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:48.402742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014275189999999999,"stat_sum":{"num_bytes":7971677,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":30,"num_read":2927,"num_read_kb":8299,"num_write":1117,"num_write_kb":6533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"75'418","reported_seq":7663,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.360797+0000","last_change":"2024-09-21T22:50:09.053339+0000","last_active":"2024-09-21T22:51:15.360797+0000","last_peered":"2024-09-21T22:51:15.360797+0000","last_clean":"2024-09-21T22:51:15.360797+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:51:15.360797+0000","last_undegraded":"2024-09-21T22:51:15.360797+0000","last_fullsized":"2024-09-21T22:51:15.360797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:46:42.716671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066547199999999996,"stat_sum":{"num_bytes":26123483,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":8879,"num_read_kb":20569,"num_write":401,"num_write_kb":5564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'462","reported_seq":5715,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258696+0000","last_change":"2024-09-21T22:50:09.062275+0000","last_active":"2024-09-21T22:51:15.258696+0000","last_peered":"2024-09-21T22:51:15.258696+0000","last_clean":"2024-09-21T22:51:15.258696+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:51:15.258696+0000","last_undegraded":"2024-09-21T22:51:15.258696+0000","last_fullsized":"2024-09-21T22:51:15.258696+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:29.274501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092760900000000003,"stat_sum":{"num_bytes":27501691,"num_objects":89,"num_object_clones":27,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":47,"num_read":5710,"num_read_kb":17021,"num_write":528,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":513821133,"num_objects":2164,"num_object_clones":619,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":1046,"num_read":159005,"num_read_kb":376547,"num_write":17282,"num_write_kb":172102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":3513975,"num_keys_recovered":33,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203456512,"data_stored":239339169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14081,"ondisk_log_size":14081,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":37204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":38096896,"data_stored":46344331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61748,"kb_used_data":34716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310092,"statfs":{"total":96636764160,"available":96573534208,"internally_reserved":0,"allocated":35549184,"data_stored":40058617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626442,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38288,"kb_used_data":6004,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94333552,"statfs":{"total":96636764160,"available":96597557248,"internally_reserved":0,"allocated":6148096,"data_stored":5717750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106128,"kb_used_data":38820,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94265712,"statfs":{"total":96636764160,"available":96528089088,"internally_reserved":0,"allocated":39751680,"data_stored":48780410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":6484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":6639616,"data_stored":8245362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56524,"kb_used_data":12856,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94315316,"statfs":{"total":96636764160,"available":96578883584,"internally_reserved":0,"allocated":13164544,"data_stored":15013672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79908,"kb_used_data":30672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94291932,"statfs":{"total":96636764160,"available":96554938368,"internally_reserved":0,"allocated":31408128,"data_stored":34638747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72400,"kb_used_data":41168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94299440,"statfs":{"total":96636764160,"available":96562626560,"internally_reserved":0,"allocated":42156032,"data_stored":46261581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29425664,"data_stored":33133123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12378112,"data_stored":14692712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":7851324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41443328,"data_stored":45962987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37675008,"data_stored":47188938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37318656,"data_stored":46030417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5263360,"data_stored":5319601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":34177024,"data_stored":39160067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:51:17.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:51:17.727 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.727 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:17.759 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T22:51:17.829 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:17.827+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:17.851 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:17.910 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:17.904+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:17.974 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:17.974 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:17.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:18.007 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:18.007 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:18.026 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:18.026 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:18.037 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:18.035+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:18.044 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:18.044 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:18.064 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:18.081 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:18.081 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:18.093 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:18.093 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:18.093 INFO:teuthology.orchestra.run.smithi094.stdout:4125 writesame 0x28b64ff thru 0x28c49f7 data_size 0x27(0xe4f9 bytes) 2024-09-21T22:51:18.098 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:18.099 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:18.104 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:18.107 INFO:teuthology.orchestra.run.smithi094.stdout:4127 trunc from 0xe3e29ac to 0xc8ce1ee 2024-09-21T22:51:18.111 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:18.104+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:18.114 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:51:18.118 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:18.118 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:18.126 INFO:teuthology.orchestra.run.smithi094.stdout:4129 write 0x55af693 thru 0x55b9f37 (0xa8a5 bytes) 2024-09-21T22:51:18.131 INFO:teuthology.orchestra.run.smithi094.stdout:4131 writesame 0x458a379 thru 0x45927f4 data_size 0x7a(0x847c bytes) 2024-09-21T22:51:18.134 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-21T22:51:17.291193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514411520,"num_objects":2168,"num_object_clones":619,"num_object_copies":4336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2168,"num_whiteouts":1046,"num_read":159121,"num_read_kb":376647,"num_write":17443,"num_write_kb":174520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":3513975,"num_keys_recovered":33,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14201,"ondisk_log_size":14201,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":521092,"kb_used_data":207924,"kb_used_omap":15,"kb_used_meta":312880,"kb_avail":754453628,"statfs":{"total":773094113280,"available":772560515072,"internally_reserved":0,"allocated":212914176,"data_stored":245060470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15556,"internal_metadata":320389948},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42516623,"num_objects":260,"num_object_clones":61,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":139,"num_read":26621,"num_read_kb":35102,"num_write":1242,"num_write_kb":9943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000672"},"pg_stats":[{"pgid":"3.1f","version":"73'855","reported_seq":6706,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.311295+0000","last_change":"2024-09-21T22:50:57.035011+0000","last_active":"2024-09-21T22:51:13.311295+0000","last_peered":"2024-09-21T22:51:13.311295+0000","last_clean":"2024-09-21T22:51:13.311295+0000","last_became_active":"2024-09-21T22:50:57.033659+0000","last_became_peered":"2024-09-21T22:50:57.033659+0000","last_unstale":"2024-09-21T22:51:13.311295+0000","last_undegraded":"2024-09-21T22:51:13.311295+0000","last_fullsized":"2024-09-21T22:51:13.311295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:02.219936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730179999999999,"stat_sum":{"num_bytes":30508004,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":32,"num_read":9055,"num_read_kb":19733,"num_write":1137,"num_write_kb":6827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":3100812,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'381","reported_seq":5564,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.454035+0000","last_change":"2024-09-21T22:50:37.035523+0000","last_active":"2024-09-21T22:51:13.454035+0000","last_peered":"2024-09-21T22:51:13.454035+0000","last_clean":"2024-09-21T22:51:13.454035+0000","last_became_active":"2024-09-21T22:50:37.030715+0000","last_became_peered":"2024-09-21T22:50:37.030715+0000","last_unstale":"2024-09-21T22:51:13.454035+0000","last_undegraded":"2024-09-21T22:51:13.454035+0000","last_fullsized":"2024-09-21T22:51:13.454035+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:02:32.907226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020037760000000001,"stat_sum":{"num_bytes":24400264,"num_objects":69,"num_object_clones":16,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":6482,"num_read_kb":18846,"num_write":349,"num_write_kb":2558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":85270,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"76'913","reported_seq":6617,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.364406+0000","last_change":"2024-09-21T22:50:09.045034+0000","last_active":"2024-09-21T22:51:16.364406+0000","last_peered":"2024-09-21T22:51:16.364406+0000","last_clean":"2024-09-21T22:51:16.364406+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:51:16.364406+0000","last_undegraded":"2024-09-21T22:51:16.364406+0000","last_fullsized":"2024-09-21T22:51:16.364406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:55.566144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010806819999999999,"stat_sum":{"num_bytes":26385297,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":9633,"num_read_kb":41238,"num_write":1399,"num_write_kb":14767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"76'308","reported_seq":1285,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279400+0000","last_change":"2024-09-21T22:50:09.046692+0000","last_active":"2024-09-21T22:51:16.279400+0000","last_peered":"2024-09-21T22:51:16.279400+0000","last_clean":"2024-09-21T22:51:16.279400+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:51:16.279400+0000","last_undegraded":"2024-09-21T22:51:16.279400+0000","last_fullsized":"2024-09-21T22:51:16.279400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:48.393370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002495611,"stat_sum":{"num_bytes":6562536,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":36,"num_read":1361,"num_read_kb":7259,"num_write":304,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"74'900","reported_seq":5062,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273989+0000","last_change":"2024-09-21T22:50:09.054590+0000","last_active":"2024-09-21T22:51:15.273989+0000","last_peered":"2024-09-21T22:51:15.273989+0000","last_clean":"2024-09-21T22:51:15.273989+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:51:15.273989+0000","last_undegraded":"2024-09-21T22:51:15.273989+0000","last_fullsized":"2024-09-21T22:51:15.273989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:10.529574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001622592,"stat_sum":{"num_bytes":7344583,"num_objects":63,"num_object_clones":22,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":7909,"num_read_kb":11374,"num_write":1435,"num_write_kb":4853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'657","reported_seq":6058,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.454418+0000","last_change":"2024-09-21T22:50:09.031950+0000","last_active":"2024-09-21T22:51:15.454418+0000","last_peered":"2024-09-21T22:51:15.454418+0000","last_clean":"2024-09-21T22:51:15.454418+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:51:15.454418+0000","last_undegraded":"2024-09-21T22:51:15.454418+0000","last_fullsized":"2024-09-21T22:51:15.454418+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:03:39.571273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002297323,"stat_sum":{"num_bytes":9569854,"num_objects":69,"num_object_clones":24,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":7502,"num_read_kb":8004,"num_write":978,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"75'376","reported_seq":4010,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266293+0000","last_change":"2024-09-21T22:50:09.032544+0000","last_active":"2024-09-21T22:51:15.266293+0000","last_peered":"2024-09-21T22:51:15.266293+0000","last_clean":"2024-09-21T22:51:15.266293+0000","last_became_active":"2024-09-21T22:50:07.033757+0000","last_became_peered":"2024-09-21T22:50:07.033757+0000","last_unstale":"2024-09-21T22:51:15.266293+0000","last_undegraded":"2024-09-21T22:51:15.266293+0000","last_fullsized":"2024-09-21T22:51:15.266293+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:38:01.192114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084097000000000002,"stat_sum":{"num_bytes":12353021,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4529,"num_read_kb":12087,"num_write":395,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"73'896","reported_seq":7978,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.457816+0000","last_change":"2024-09-21T22:50:09.027841+0000","last_active":"2024-09-21T22:51:13.457816+0000","last_peered":"2024-09-21T22:51:13.457816+0000","last_clean":"2024-09-21T22:51:13.457816+0000","last_became_active":"2024-09-21T22:50:07.034008+0000","last_became_peered":"2024-09-21T22:50:07.034008+0000","last_unstale":"2024-09-21T22:51:13.457816+0000","last_undegraded":"2024-09-21T22:51:13.457816+0000","last_fullsized":"2024-09-21T22:51:13.457816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:21.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020218739999999999,"stat_sum":{"num_bytes":19733066,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":11049,"num_read_kb":20725,"num_write":1527,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"74'266","reported_seq":2126,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273845+0000","last_change":"2024-09-21T22:50:09.054285+0000","last_active":"2024-09-21T22:51:15.273845+0000","last_peered":"2024-09-21T22:51:15.273845+0000","last_clean":"2024-09-21T22:51:15.273845+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:51:15.273845+0000","last_undegraded":"2024-09-21T22:51:15.273845+0000","last_fullsized":"2024-09-21T22:51:15.273845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:12.184069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013918940000000001,"stat_sum":{"num_bytes":6160592,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":35,"num_read":1916,"num_read_kb":4004,"num_write":267,"num_write_kb":2440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"76'295","reported_seq":6706,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.424291+0000","last_change":"2024-09-21T22:50:09.045475+0000","last_active":"2024-09-21T22:51:16.424291+0000","last_peered":"2024-09-21T22:51:16.424291+0000","last_clean":"2024-09-21T22:51:16.424291+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:51:16.424291+0000","last_undegraded":"2024-09-21T22:51:16.424291+0000","last_fullsized":"2024-09-21T22:51:16.424291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:49.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504238,"stat_sum":{"num_bytes":18228670,"num_objects":80,"num_object_clones":24,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":6371,"num_read_kb":6196,"num_write":256,"num_write_kb":3667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"74'308","reported_seq":10653,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258662+0000","last_change":"2024-09-21T22:50:09.062094+0000","last_active":"2024-09-21T22:51:15.258662+0000","last_peered":"2024-09-21T22:51:15.258662+0000","last_clean":"2024-09-21T22:51:15.258662+0000","last_became_active":"2024-09-21T22:50:07.039164+0000","last_became_peered":"2024-09-21T22:50:07.039164+0000","last_unstale":"2024-09-21T22:51:15.258662+0000","last_undegraded":"2024-09-21T22:51:15.258662+0000","last_fullsized":"2024-09-21T22:51:15.258662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:44.821161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081156800000000003,"stat_sum":{"num_bytes":34643350,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":10646,"num_read_kb":37673,"num_write":301,"num_write_kb":15178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"74'387","reported_seq":6287,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.271524+0000","last_change":"2024-09-21T22:50:09.032016+0000","last_active":"2024-09-21T22:51:15.271524+0000","last_peered":"2024-09-21T22:51:15.271524+0000","last_clean":"2024-09-21T22:51:15.271524+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:51:15.271524+0000","last_undegraded":"2024-09-21T22:51:15.271524+0000","last_fullsized":"2024-09-21T22:51:15.271524+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:46.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021866500000000001,"stat_sum":{"num_bytes":22342776,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":32,"num_read":6462,"num_read_kb":9579,"num_write":462,"num_write_kb":9838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.432801+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:51:16.432801+0000","last_peered":"2024-09-21T22:51:16.432801+0000","last_clean":"2024-09-21T22:51:16.432801+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:51:16.432801+0000","last_undegraded":"2024-09-21T22:51:16.432801+0000","last_fullsized":"2024-09-21T22:51:16.432801+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"74'284","reported_seq":5965,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.273848+0000","last_change":"2024-09-21T22:50:09.053734+0000","last_active":"2024-09-21T22:51:15.273848+0000","last_peered":"2024-09-21T22:51:15.273848+0000","last_clean":"2024-09-21T22:51:15.273848+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:51:15.273848+0000","last_undegraded":"2024-09-21T22:51:15.273848+0000","last_fullsized":"2024-09-21T22:51:15.273848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:29.531204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003049,"stat_sum":{"num_bytes":17136162,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":5634,"num_read_kb":13409,"num_write":281,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266348+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:51:15.266348+0000","last_peered":"2024-09-21T22:51:15.266348+0000","last_clean":"2024-09-21T22:51:15.266348+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:51:15.266348+0000","last_undegraded":"2024-09-21T22:51:15.266348+0000","last_fullsized":"2024-09-21T22:51:15.266348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"73'400","reported_seq":1861,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.457646+0000","last_change":"2024-09-21T22:50:09.027492+0000","last_active":"2024-09-21T22:51:13.457646+0000","last_peered":"2024-09-21T22:51:13.457646+0000","last_clean":"2024-09-21T22:51:13.457646+0000","last_became_active":"2024-09-21T22:50:07.028763+0000","last_became_peered":"2024-09-21T22:50:07.028763+0000","last_unstale":"2024-09-21T22:51:13.457646+0000","last_undegraded":"2024-09-21T22:51:13.457646+0000","last_fullsized":"2024-09-21T22:51:13.457646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:39.923572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018249729999999999,"stat_sum":{"num_bytes":5871212,"num_objects":67,"num_object_clones":21,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":2537,"num_read_kb":4130,"num_write":462,"num_write_kb":2736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258755+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:51:15.258755+0000","last_peered":"2024-09-21T22:51:15.258755+0000","last_clean":"2024-09-21T22:51:15.258755+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:51:15.258755+0000","last_undegraded":"2024-09-21T22:51:15.258755+0000","last_fullsized":"2024-09-21T22:51:15.258755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'489","reported_seq":2236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266378+0000","last_change":"2024-09-21T22:50:09.032823+0000","last_active":"2024-09-21T22:51:15.266378+0000","last_peered":"2024-09-21T22:51:15.266378+0000","last_clean":"2024-09-21T22:51:15.266378+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:51:15.266378+0000","last_undegraded":"2024-09-21T22:51:15.266378+0000","last_fullsized":"2024-09-21T22:51:15.266378+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:04:18.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093496699999999998,"stat_sum":{"num_bytes":8336757,"num_objects":73,"num_object_clones":24,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":2721,"num_read_kb":2833,"num_write":610,"num_write_kb":3557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.266349+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:51:15.266349+0000","last_peered":"2024-09-21T22:51:15.266349+0000","last_clean":"2024-09-21T22:51:15.266349+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:51:15.266349+0000","last_undegraded":"2024-09-21T22:51:15.266349+0000","last_fullsized":"2024-09-21T22:51:15.266349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'229","reported_seq":751,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279946+0000","last_change":"2024-09-21T22:50:09.046691+0000","last_active":"2024-09-21T22:51:16.279946+0000","last_peered":"2024-09-21T22:51:16.279946+0000","last_clean":"2024-09-21T22:51:16.279946+0000","last_became_active":"2024-09-21T22:50:07.039630+0000","last_became_peered":"2024-09-21T22:50:07.039630+0000","last_unstale":"2024-09-21T22:51:16.279946+0000","last_undegraded":"2024-09-21T22:51:16.279946+0000","last_fullsized":"2024-09-21T22:51:16.279946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:22:56.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021691330000000002,"stat_sum":{"num_bytes":8327811,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":457,"num_read_kb":9110,"num_write":258,"num_write_kb":7708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'350","reported_seq":3838,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.271602+0000","last_change":"2024-09-21T22:50:09.030568+0000","last_active":"2024-09-21T22:51:15.271602+0000","last_peered":"2024-09-21T22:51:15.271602+0000","last_clean":"2024-09-21T22:51:15.271602+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:51:15.271602+0000","last_undegraded":"2024-09-21T22:51:15.271602+0000","last_fullsized":"2024-09-21T22:51:15.271602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:19.849107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079408,"stat_sum":{"num_bytes":15531119,"num_objects":63,"num_object_clones":21,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":4541,"num_read_kb":8357,"num_write":336,"num_write_kb":5019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279188+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:51:16.279188+0000","last_peered":"2024-09-21T22:51:16.279188+0000","last_clean":"2024-09-21T22:51:16.279188+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:51:16.279188+0000","last_undegraded":"2024-09-21T22:51:16.279188+0000","last_fullsized":"2024-09-21T22:51:16.279188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"75'487","reported_seq":4346,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.400774+0000","last_change":"2024-09-21T22:50:09.032584+0000","last_active":"2024-09-21T22:51:15.400774+0000","last_peered":"2024-09-21T22:51:15.400774+0000","last_clean":"2024-09-21T22:51:15.400774+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:51:15.400774+0000","last_undegraded":"2024-09-21T22:51:15.400774+0000","last_fullsized":"2024-09-21T22:51:15.400774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:16:05.104110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076847199999999997,"stat_sum":{"num_bytes":24377689,"num_objects":79,"num_object_clones":27,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":37,"num_read":5446,"num_read_kb":15899,"num_write":735,"num_write_kb":9286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258435+0000","last_change":"2024-09-21T22:49:59.203552+0000","last_active":"2024-09-21T22:51:15.258435+0000","last_peered":"2024-09-21T22:51:15.258435+0000","last_clean":"2024-09-21T22:51:15.258435+0000","last_became_active":"2024-09-21T22:49:57.287639+0000","last_became_peered":"2024-09-21T22:49:57.287639+0000","last_unstale":"2024-09-21T22:51:15.258435+0000","last_undegraded":"2024-09-21T22:51:15.258435+0000","last_fullsized":"2024-09-21T22:51:15.258435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:03.915894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"75'283","reported_seq":1299,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.425459+0000","last_change":"2024-09-21T22:50:09.053334+0000","last_active":"2024-09-21T22:51:15.425459+0000","last_peered":"2024-09-21T22:51:15.425459+0000","last_clean":"2024-09-21T22:51:15.425459+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:51:15.425459+0000","last_undegraded":"2024-09-21T22:51:15.425459+0000","last_fullsized":"2024-09-21T22:51:15.425459+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:04.019239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068683399999999999,"stat_sum":{"num_bytes":15887827,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":37,"num_read":1021,"num_read_kb":4914,"num_write":247,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258852+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:51:15.258852+0000","last_peered":"2024-09-21T22:51:15.258852+0000","last_clean":"2024-09-21T22:51:15.258852+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:51:15.258852+0000","last_undegraded":"2024-09-21T22:51:15.258852+0000","last_fullsized":"2024-09-21T22:51:15.258852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"71'492","reported_seq":1975,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.252923+0000","last_change":"2024-09-21T22:50:09.026491+0000","last_active":"2024-09-21T22:51:13.252923+0000","last_peered":"2024-09-21T22:51:13.252923+0000","last_clean":"2024-09-21T22:51:13.252923+0000","last_became_active":"2024-09-21T22:50:07.028774+0000","last_became_peered":"2024-09-21T22:50:07.028774+0000","last_unstale":"2024-09-21T22:51:13.252923+0000","last_undegraded":"2024-09-21T22:51:13.252923+0000","last_fullsized":"2024-09-21T22:51:13.252923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:37:30.581308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010115009999999999,"stat_sum":{"num_bytes":11862562,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":28,"num_read":2570,"num_read_kb":6859,"num_write":657,"num_write_kb":5385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279219+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:51:16.279219+0000","last_peered":"2024-09-21T22:51:16.279219+0000","last_clean":"2024-09-21T22:51:16.279219+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:51:16.279219+0000","last_undegraded":"2024-09-21T22:51:16.279219+0000","last_fullsized":"2024-09-21T22:51:16.279219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258624+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:51:15.258624+0000","last_peered":"2024-09-21T22:51:15.258624+0000","last_clean":"2024-09-21T22:51:15.258624+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:51:15.258624+0000","last_undegraded":"2024-09-21T22:51:15.258624+0000","last_fullsized":"2024-09-21T22:51:15.258624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"73'262","reported_seq":2013,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.313501+0000","last_change":"2024-09-21T22:50:09.026886+0000","last_active":"2024-09-21T22:51:13.313501+0000","last_peered":"2024-09-21T22:51:13.313501+0000","last_clean":"2024-09-21T22:51:13.313501+0000","last_became_active":"2024-09-21T22:50:07.032070+0000","last_became_peered":"2024-09-21T22:50:07.032070+0000","last_unstale":"2024-09-21T22:51:13.313501+0000","last_undegraded":"2024-09-21T22:51:13.313501+0000","last_fullsized":"2024-09-21T22:51:13.313501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:11.597978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001477641,"stat_sum":{"num_bytes":12396492,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":2144,"num_read_kb":2316,"num_write":254,"num_write_kb":1297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"75'244","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.429986+0000","last_change":"2024-09-21T22:50:09.030512+0000","last_active":"2024-09-21T22:51:15.429986+0000","last_peered":"2024-09-21T22:51:15.429986+0000","last_clean":"2024-09-21T22:51:15.429986+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:51:15.429986+0000","last_undegraded":"2024-09-21T22:51:15.429986+0000","last_fullsized":"2024-09-21T22:51:15.429986+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:11.890240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00126947,"stat_sum":{"num_bytes":6511302,"num_objects":64,"num_object_clones":19,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":37,"num_read":3354,"num_read_kb":4212,"num_write":204,"num_write_kb":1562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"76'215","reported_seq":633,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.485129+0000","last_change":"2024-09-21T22:50:34.240149+0000","last_active":"2024-09-21T22:51:16.485129+0000","last_peered":"2024-09-21T22:51:16.485129+0000","last_clean":"2024-09-21T22:51:16.485129+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:51:16.485129+0000","last_undegraded":"2024-09-21T22:51:16.485129+0000","last_fullsized":"2024-09-21T22:51:16.485129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:29:39.305976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092677,"stat_sum":{"num_bytes":5339608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":409,"num_read_kb":1322,"num_write":202,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"72'256","reported_seq":1232,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T22:51:13.253039+0000","last_change":"2024-09-21T22:50:09.027738+0000","last_active":"2024-09-21T22:51:13.253039+0000","last_peered":"2024-09-21T22:51:13.253039+0000","last_clean":"2024-09-21T22:51:13.253039+0000","last_became_active":"2024-09-21T22:50:07.037670+0000","last_became_peered":"2024-09-21T22:50:07.037670+0000","last_unstale":"2024-09-21T22:51:13.253039+0000","last_undegraded":"2024-09-21T22:51:13.253039+0000","last_fullsized":"2024-09-21T22:51:13.253039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:02.862433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002153528,"stat_sum":{"num_bytes":10954015,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":30,"num_read":925,"num_read_kb":4705,"num_write":228,"num_write_kb":4583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"75'266","reported_seq":4981,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.279296+0000","last_change":"2024-09-21T22:50:09.044903+0000","last_active":"2024-09-21T22:51:16.279296+0000","last_peered":"2024-09-21T22:51:16.279296+0000","last_clean":"2024-09-21T22:51:16.279296+0000","last_became_active":"2024-09-21T22:50:07.035007+0000","last_became_peered":"2024-09-21T22:50:07.035007+0000","last_unstale":"2024-09-21T22:51:16.279296+0000","last_undegraded":"2024-09-21T22:51:16.279296+0000","last_fullsized":"2024-09-21T22:51:16.279296+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:29:32.095871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261420000000001,"stat_sum":{"num_bytes":18079611,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":4822,"num_read_kb":13795,"num_write":285,"num_write_kb":5826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"74'435","reported_seq":2815,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258714+0000","last_change":"2024-09-21T22:50:09.064319+0000","last_active":"2024-09-21T22:51:15.258714+0000","last_peered":"2024-09-21T22:51:15.258714+0000","last_clean":"2024-09-21T22:51:15.258714+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:51:15.258714+0000","last_undegraded":"2024-09-21T22:51:15.258714+0000","last_fullsized":"2024-09-21T22:51:15.258714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:52:11.181511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024500839999999999,"stat_sum":{"num_bytes":6906500,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":4507,"num_read_kb":7629,"num_write":421,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"75'349","reported_seq":4162,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.402976+0000","last_change":"2024-09-21T22:50:09.064534+0000","last_active":"2024-09-21T22:51:15.402976+0000","last_peered":"2024-09-21T22:51:15.402976+0000","last_clean":"2024-09-21T22:51:15.402976+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:51:15.402976+0000","last_undegraded":"2024-09-21T22:51:15.402976+0000","last_fullsized":"2024-09-21T22:51:15.402976+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:53:34.466465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002506234,"stat_sum":{"num_bytes":18508847,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":4543,"num_read_kb":8378,"num_write":399,"num_write_kb":4576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"74'377","reported_seq":4863,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258901+0000","last_change":"2024-09-21T22:50:09.064820+0000","last_active":"2024-09-21T22:51:15.258901+0000","last_peered":"2024-09-21T22:51:15.258901+0000","last_clean":"2024-09-21T22:51:15.258901+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:51:15.258901+0000","last_undegraded":"2024-09-21T22:51:15.258901+0000","last_fullsized":"2024-09-21T22:51:15.258901+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:27.338219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021047100000000001,"stat_sum":{"num_bytes":27316827,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5761,"num_read_kb":8654,"num_write":418,"num_write_kb":2651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"76'438","reported_seq":6192,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T22:51:16.536991+0000","last_change":"2024-09-21T22:50:37.035011+0000","last_active":"2024-09-21T22:51:16.536991+0000","last_peered":"2024-09-21T22:51:16.536991+0000","last_clean":"2024-09-21T22:51:16.536991+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:51:16.536991+0000","last_undegraded":"2024-09-21T22:51:16.536991+0000","last_fullsized":"2024-09-21T22:51:16.536991+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:02.613006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001972101,"stat_sum":{"num_bytes":20647928,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":38,"num_read":7182,"num_read_kb":17418,"num_write":429,"num_write_kb":6292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"75'803","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258663+0000","last_change":"2024-09-21T22:50:09.062805+0000","last_active":"2024-09-21T22:51:15.258663+0000","last_peered":"2024-09-21T22:51:15.258663+0000","last_clean":"2024-09-21T22:51:15.258663+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:51:15.258663+0000","last_undegraded":"2024-09-21T22:51:15.258663+0000","last_fullsized":"2024-09-21T22:51:15.258663+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:48.402742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014275189999999999,"stat_sum":{"num_bytes":7971677,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":30,"num_read":2927,"num_read_kb":8299,"num_write":1117,"num_write_kb":6533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"75'418","reported_seq":7663,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.360797+0000","last_change":"2024-09-21T22:50:09.053339+0000","last_active":"2024-09-21T22:51:15.360797+0000","last_peered":"2024-09-21T22:51:15.360797+0000","last_clean":"2024-09-21T22:51:15.360797+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:51:15.360797+0000","last_undegraded":"2024-09-21T22:51:15.360797+0000","last_fullsized":"2024-09-21T22:51:15.360797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:46:42.716671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066547199999999996,"stat_sum":{"num_bytes":26123483,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":8879,"num_read_kb":20569,"num_write":401,"num_write_kb":5564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'462","reported_seq":5715,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T22:51:15.258696+0000","last_change":"2024-09-21T22:50:09.062275+0000","last_active":"2024-09-21T22:51:15.258696+0000","last_peered":"2024-09-21T22:51:15.258696+0000","last_clean":"2024-09-21T22:51:15.258696+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:51:15.258696+0000","last_undegraded":"2024-09-21T22:51:15.258696+0000","last_fullsized":"2024-09-21T22:51:15.258696+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:29.274501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092760900000000003,"stat_sum":{"num_bytes":27501691,"num_objects":89,"num_object_clones":27,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":47,"num_read":5710,"num_read_kb":17021,"num_write":528,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":513821133,"num_objects":2164,"num_object_clones":619,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":1046,"num_read":159005,"num_read_kb":376547,"num_write":17282,"num_write_kb":172102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":3513975,"num_keys_recovered":33,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203456512,"data_stored":239339169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14081,"ondisk_log_size":14081,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":37204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":38096896,"data_stored":46344331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61748,"kb_used_data":34716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310092,"statfs":{"total":96636764160,"available":96573534208,"internally_reserved":0,"allocated":35549184,"data_stored":40058617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626442,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38288,"kb_used_data":6004,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94333552,"statfs":{"total":96636764160,"available":96597557248,"internally_reserved":0,"allocated":6148096,"data_stored":5717750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106128,"kb_used_data":38820,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94265712,"statfs":{"total":96636764160,"available":96528089088,"internally_reserved":0,"allocated":39751680,"data_stored":48780410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":6484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":6639616,"data_stored":8245362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56524,"kb_used_data":12856,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94315316,"statfs":{"total":96636764160,"available":96578883584,"internally_reserved":0,"allocated":13164544,"data_stored":15013672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79908,"kb_used_data":30672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94291932,"statfs":{"total":96636764160,"available":96554938368,"internally_reserved":0,"allocated":31408128,"data_stored":34638747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72400,"kb_used_data":41168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94299440,"statfs":{"total":96636764160,"available":96562626560,"internally_reserved":0,"allocated":42156032,"data_stored":46261581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29425664,"data_stored":33133123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12378112,"data_stored":14692712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":7851324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41443328,"data_stored":45962987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37675008,"data_stored":47188938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37318656,"data_stored":46030417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5263360,"data_stored":5319601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":34177024,"data_stored":39160067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:51:18.136 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T22:51:18.136 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:51:18.136 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-21T22:51:18.136 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T22:51:18.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:51:18.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:18.137 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:18.139 INFO:teuthology.orchestra.run.smithi094.stdout:4132 write 0x9597635 thru 0x95a00c0 (0x8a8c bytes) 2024-09-21T22:51:18.145 INFO:teuthology.orchestra.run.smithi094.stdout:4135 punch from 0x65480d4 to 0x65508e4, (0x8810 bytes) 2024-09-21T22:51:18.145 INFO:teuthology.orchestra.run.smithi094.stdout:4136 read 0xa9b6244 thru 0xa9b738c (0x1149 bytes) 2024-09-21T22:51:18.145 INFO:teuthology.orchestra.run.smithi094.stdout:4139 read 0x54d797a thru 0x54e635d (0xe9e4 bytes) 2024-09-21T22:51:18.145 INFO:teuthology.orchestra.run.smithi094.stdout:4141 write 0xaefcc03 thru 0xaf002aa (0x36a8 bytes) 2024-09-21T22:51:18.154 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:18.155 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:18.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:18.212 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:18.208+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:18.231 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T22:51:18.231 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-21T22:51:18.231 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:51:18.232 DEBUG:teuthology.orchestra.run.smithi094:> type ceph-objectstore-tool 2024-09-21T22:51:18.271 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:18.272 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:18.276 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T22:51:18.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-21T22:51:18.284 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:51:18.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:18.348 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:18.344+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.371737133000003, 2024-09-21T22:51:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.367499563999999, 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.367304666000003, 2024-09-21T22:51:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68484 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 356352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752757928999999, 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033448665000000002, 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68484 2024-09-21T22:51:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638315+0000", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638316+0000", 2024-09-21T22:51:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0100000000000001e-06 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638322+0000", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169999999999997e-06 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638342+0000", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.995e-05 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638370+0000", 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7985000000000001e-05 2024-09-21T22:51:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671391+0000", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033021259999999997 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671483+0000", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1380999999999998e-05 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671763+0000", 2024-09-21T22:51:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028066199999999999 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68541 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 0~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752682221000001, 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033895258999999997, 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68541 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638390+0000", 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638391+0000", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638398+0000", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9770000000000003e-06 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638421+0000", 2024-09-21T22:51:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2578999999999998e-05 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638436+0000", 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4620000000000001e-05 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671874+0000", 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033438273999999997 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.671995+0000", 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120567 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672286+0000", 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029103199999999999 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68542 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 49152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752617623000001, 2024-09-21T22:51:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034317184000000001, 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68542 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638455+0000", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638456+0000", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.567e-06 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638462+0000", 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9830000000000001e-06 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638487+0000", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4199999999999999e-05 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638501+0000", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4489e-05 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672333+0000", 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033832167000000003 2024-09-21T22:51:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672439+0000", 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105635 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672772+0000", 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033314299999999999 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68543 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 81920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752552334000001, 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034634933999999999, 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68543 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638520+0000", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638521+0000", 2024-09-21T22:51:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1060000000000001e-06 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638527+0000", 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638546+0000", 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8635999999999999e-05 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638559+0000", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2733e-05 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672814+0000", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034255173 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.672909+0000", 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4581999999999997e-05 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673155+0000", 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024655400000000001 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68544 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 163840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752494999, 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034985167999999997, 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68544 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638578+0000", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638579+0000", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006e-06 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638584+0000", 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6840000000000001e-06 2024-09-21T22:51:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638604+0000", 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9859000000000001e-05 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638618+0000", 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.428e-05 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673193+0000", 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034575074999999997 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673252+0000", 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8288000000000002e-05 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673563+0000", 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031097600000000003 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68545 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 196608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752356130000001, 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035408569000000001, 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68545 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638716+0000", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638718+0000", 2024-09-21T22:51:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3030000000000001e-06 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638725+0000", 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8299999999999998e-06 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638746+0000", 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1566e-05 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638760+0000", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673630+0000", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034869372000000003 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.673803+0000", 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000173435 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674125+0000", 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032173299999999999 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68546 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 245760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752280044999999, 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035763809000000001, 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68546 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638792+0000", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638794+0000", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4529999999999999e-06 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638815+0000", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1418999999999999e-05 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638837+0000", 2024-09-21T22:51:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1855999999999999e-05 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638851+0000", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.341e-05 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674168+0000", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035317861999999998 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674234+0000", 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5946999999999997e-05 2024-09-21T22:51:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674556+0000", 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032186200000000002 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68547 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 278528~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752199508, 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03614469, 2024-09-21T22:51:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68547 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638873+0000", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638874+0000", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638881+0000", 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9999999999999999e-06 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638901+0000", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999999e-05 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638915+0000", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4532000000000001e-05 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674611+0000", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035696033000000002 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.674712+0000", 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010040299999999999 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675018+0000", 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030589199999999999 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68548 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752138782999999, 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036468652999999997, 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68548 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638934+0000", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638935+0000", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638941+0000", 2024-09-21T22:51:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638959+0000", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8431999999999999e-05 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638973+0000", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3813e-05 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675083+0000", 2024-09-21T22:51:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036110541000000003 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675199+0000", 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115926 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675402+0000", 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020310400000000001 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68605 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.752081605000001, 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036815352000000003, 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68605 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638991+0000", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638992+0000", 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.638998+0000", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4370000000000004e-06 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639017+0000", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9018999999999999e-05 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639055+0000", 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7441999999999998e-05 2024-09-21T22:51:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675453+0000", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036398463999999998 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675531+0000", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7575000000000002e-05 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675806+0000", 2024-09-21T22:51:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027533800000000003 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68606 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.7519969, 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036991366999999997, 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68606 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639076+0000", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639077+0000", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1829999999999999e-06 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639085+0000", 2024-09-21T22:51:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7559999999999999e-06 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639118+0000", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3705999999999999e-05 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639148+0000", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0162999999999999e-05 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675849+0000", 2024-09-21T22:51:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036700993000000001 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.675907+0000", 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8032999999999998e-05 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676067+0000", 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015953300000000001 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68607 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 86016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.751902883, 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037187378, 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68607 2024-09-21T22:51:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639170+0000", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639171+0000", 2024-09-21T22:51:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639178+0000", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6699999999999997e-06 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639198+0000", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0132e-05 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639212+0000", 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4294000000000001e-05 2024-09-21T22:51:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676124+0000", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036912568 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676193+0000", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8821999999999999e-05 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676357+0000", 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016368200000000001 2024-09-21T22:51:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68608 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.751838872, 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037593025000000002, 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68608 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639234+0000", 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639235+0000", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.034e-06 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639241+0000", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5830000000000003e-06 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639268+0000", 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6741999999999999e-05 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639281+0000", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.344e-05 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676415+0000", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037133806999999998 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676541+0000", 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012614500000000001 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676827+0000", 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028527400000000002 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68609 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.751771657000001, 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037858175000000001, 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68609 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639301+0000", 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639302+0000", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0669999999999999e-06 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639309+0000", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7429999999999996e-06 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639327+0000", 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8573e-05 2024-09-21T22:51:18.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639341+0000", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3286e-05 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676887+0000", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037546698000000003 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.676966+0000", 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8611999999999999e-05 2024-09-21T22:51:18.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677159+0000", 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000193196 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68610 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.751713208, 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038202400999999997, 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68610 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639359+0000", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639360+0000", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639367+0000", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1759999999999998e-06 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639385+0000", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8454e-05 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639399+0000", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3835999999999999e-05 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677208+0000", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037809295999999999 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677317+0000", 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108682 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677562+0000", 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024489100000000001 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68611 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.751654739999999, 2024-09-21T22:51:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038561235999999999, 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68611 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639418+0000", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639419+0000", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.077e-06 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639425+0000", 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639443+0000", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8675999999999999e-05 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639457+0000", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4083000000000001e-05 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677605+0000", 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038147542999999999 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677713+0000", 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108529 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.677979+0000", 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026561500000000002 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:68612 3.10 3:0853d65b:::rbd_data.111e28e769d9.0000000000000005:17 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T22:51:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.75156651, 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038912731999999998, 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 68612 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:18.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639506+0000", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639508+0000", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:18.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639514+0000", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3500000000000002e-06 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639534+0000", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0199000000000001e-05 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.639549+0000", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4056000000000001e-05 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:18.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678034+0000", 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.038485761 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678166+0000", 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131719 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:04.678419+0000", 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025270300000000001 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:18.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:18.488 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:18.489 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:18.483+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:18.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:18.590 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:18.587+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3.9ac0f84b (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.33132325800000001, 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.33136847400000002, 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.617 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:18.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:51:18.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:51:18.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:18.621 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 1 2024-09-21T22:51:18.622 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:18.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:18.691 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:18.684+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:18.747 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:18.747 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:18.748 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:18.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:18.792 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:18.787+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:18.893 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:18.891+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37436 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 342016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.434483143000001, 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069292702999999997, 2024-09-21T22:51:18.915 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37436 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479223+0000", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479235+0000", 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1800000000000001e-05 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479246+0000", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0407e-05 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479253+0000", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.547962+0000", 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068709281999999997 2024-09-21T22:51:18.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548045+0000", 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3084000000000006e-05 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548516+0000", 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047039 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37437 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 352256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.434443512999998, 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069486928000000003, 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37437 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479263+0000", 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7699999999999992e-07 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479268+0000", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9099999999999996e-06 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479285+0000", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6993000000000001e-05 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.479299+0000", 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3873000000000001e-05 2024-09-21T22:51:18.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548552+0000", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069252805000000001 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548641+0000", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9262e-05 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548749+0000", 2024-09-21T22:51:18.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108108 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37439 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 362496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.433678028000003, 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069180205999999994, 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37439 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480028+0000", 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480030+0000", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480036+0000", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480056+0000", 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9983000000000002e-05 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480071+0000", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4226000000000001e-05 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548792+0000", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068721029000000003 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.548882+0000", 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9963999999999994e-05 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549208+0000", 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000326554 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37441 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 387072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.43346408, 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069300743999999997, 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37441 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480242+0000", 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480243+0000", 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480252+0000", 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2060000000000007e-06 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480271+0000", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9357000000000001e-05 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480283+0000", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2193e-05 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549240+0000", 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068956442000000007 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549323+0000", 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3024999999999999e-05 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549543+0000", 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000220027 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37443 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.932 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.433374086999997, 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069462802000000004, 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37443 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:18.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480332+0000", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480333+0000", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5300000000000003e-07 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480336+0000", 2024-09-21T22:51:18.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6370000000000001e-06 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480346+0000", 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7229999999999999e-06 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480353+0000", 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1230000000000002e-06 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549595+0000", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069241493000000001 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549684+0000", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8919999999999996e-05 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549795+0000", 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111153 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37445 3.1b 3:d8c07738:::rbd_data.111ebb10c685.0000000000000001:e [read 421888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.433312870999998, 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069665381999999998, 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37445 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.937 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480393+0000", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480394+0000", 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8400000000000004e-07 2024-09-21T22:51:18.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480397+0000", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2100000000000002e-06 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480427+0000", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0153000000000001e-05 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.480441+0000", 2024-09-21T22:51:18.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4283e-05 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549856+0000", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069414044999999994 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.549944+0000", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8485000000000004e-05 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.550059+0000", 2024-09-21T22:51:18.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114522 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40413 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 55296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872471501, 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068809408000000002, 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40413 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041235+0000", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1800000000000002e-07 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041236+0000", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5799999999999999e-06 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041241+0000", 2024-09-21T22:51:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4649999999999996e-06 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041244+0000", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109938+0000", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068693828999999998 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.109989+0000", 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1060000000000002e-05 2024-09-21T22:51:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110044+0000", 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4960999999999998e-05 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40414 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872457954000001, 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068921915, 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40414 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041248+0000", 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1199999999999999e-07 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041253+0000", 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1829999999999996e-06 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041259+0000", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4409999999999998e-06 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041264+0000", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110071+0000", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068807726 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110117+0000", 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5164999999999999e-05 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110170+0000", 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3368999999999997e-05 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40415 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 67584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872437571999999, 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069034874999999996, 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40415 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041268+0000", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041269+0000", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041270+0000", 2024-09-21T22:51:18.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041275+0000", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2849999999999996e-06 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041278+0000", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1609999999999999e-06 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110197+0000", 2024-09-21T22:51:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068919451000000007 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110249+0000", 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1703999999999998e-05 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110303+0000", 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4259999999999999e-05 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40416 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 75776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872424143, 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069146221999999993, 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40416 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041282+0000", 2024-09-21T22:51:18.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5600000000000002e-07 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041284+0000", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7290000000000001e-06 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041289+0000", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3059999999999997e-06 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041293+0000", 2024-09-21T22:51:18.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110331+0000", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069038424000000001 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110374+0000", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3198999999999998e-05 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110428+0000", 2024-09-21T22:51:18.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3958000000000001e-05 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40417 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872409680000001, 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069265832999999999, 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.958 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40417 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041296+0000", 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041297+0000", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7700000000000001e-07 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041298+0000", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.387e-06 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041302+0000", 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0570000000000003e-06 2024-09-21T22:51:18.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041305+0000", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.943e-06 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110457+0000", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069151578000000005 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110503+0000", 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6184999999999999e-05 2024-09-21T22:51:18.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110562+0000", 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9406e-05 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40418 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 88064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872397055, 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069472528000000006, 2024-09-21T22:51:18.962 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40418 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041309+0000", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041310+0000", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-07 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041311+0000", 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3149999999999999e-06 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041315+0000", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4739999999999999e-06 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041319+0000", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2370000000000001e-06 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110590+0000", 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069271505999999997 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110674+0000", 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3594e-05 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110781+0000", 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107799 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40419 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.966 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872358654999999, 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069592774999999996, 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40419 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:18.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041347+0000", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041348+0000", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3399999999999999e-07 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041356+0000", 2024-09-21T22:51:18.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000001e-06 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041363+0000", 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0840000000000003e-06 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041366+0000", 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5769999999999998e-06 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110815+0000", 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069448787999999997 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110876+0000", 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0554000000000003e-05 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110940+0000", 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4537999999999998e-05 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40420 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 100352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872334183, 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069702782000000005, 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40420 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.971 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041372+0000", 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9599999999999999e-07 2024-09-21T22:51:18.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041374+0000", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.547e-06 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041378+0000", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2760000000000002e-06 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041386+0000", 2024-09-21T22:51:18.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7780000000000007e-06 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.110973+0000", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069586604999999996 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111019+0000", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6587e-05 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111075+0000", 2024-09-21T22:51:18.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5492999999999999e-05 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40421 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 108544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872315317999998, 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069829363000000005, 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.975 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40421 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041391+0000", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9499999999999998e-07 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041393+0000", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5549999999999999e-06 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041397+0000", 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041400+0000", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1949999999999999e-06 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111104+0000", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069703782000000006 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111157+0000", 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2809999999999998e-05 2024-09-21T22:51:18.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111220+0000", 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3357999999999999e-05 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40422 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 114688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872302081000001, 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069947285999999997, 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40422 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041404+0000", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7599999999999998e-07 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041406+0000", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3650000000000001e-06 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041410+0000", 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1609999999999996e-06 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041413+0000", 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111251+0000", 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069838064000000005 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111295+0000", 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4490000000000003e-05 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111351+0000", 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5912999999999998e-05 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.983 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40423 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 120832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872289108, 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070071562000000004, 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40423 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041417+0000", 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5100000000000001e-07 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041418+0000", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041423+0000", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479999999999999e-06 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041426+0000", 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3519999999999998e-06 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111379+0000", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069952997000000003 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111431+0000", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1743999999999998e-05 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111488+0000", 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7800000000000002e-05 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40424 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872276314, 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07021078, 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40424 2024-09-21T22:51:18.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041430+0000", 2024-09-21T22:51:18.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.53e-07 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041431+0000", 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4279999999999999e-06 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041435+0000", 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0960000000000003e-06 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041439+0000", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1109999999999999e-06 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111539+0000", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070100649000000001 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111584+0000", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4750999999999999e-05 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111640+0000", 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6492000000000001e-05 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40425 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 133120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.872263813, 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.070346305999999997, 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40425 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041442+0000", 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041727+0000", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028498199999999998 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041731+0000", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5099999999999999e-06 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041739+0000", 2024-09-21T22:51:18.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1550000000000007e-06 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041744+0000", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000002e-06 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111671+0000", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069927538999999997 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111728+0000", 2024-09-21T22:51:18.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6403000000000002e-05 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111789+0000", 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0877e-05 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:40426 3.1f 3:f936df87:::rbd_data.111e4aac08c7.000000000000007e:10 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.871953387000001, 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07016704, 2024-09-21T22:51:18.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 40426 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.041753+0000", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044582+0000", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028291309999999999 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044600+0000", 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7856999999999999e-05 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044612+0000", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2754e-05 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.044619+0000", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6440000000000003e-06 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111818+0000", 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.067199470999999997 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:18.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111862+0000", 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3695000000000001e-05 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:47.111920+0000", 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7488000000000003e-05 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.000 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.002 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:19.002 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:18.996+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:19.102 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:19.099+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:19.152 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.152 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:19.152 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:19.153 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:19.202 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:19.199+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:19.285 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.285 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:19.285 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:19.285 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:19.285 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.295 INFO:teuthology.orchestra.run.smithi094.stdout:4142 write 0x7716e0 thru 0x77e7ae (0xd0cf bytes) 2024-09-21T22:51:19.303 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:19.299+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:19.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:19.340 INFO:teuthology.orchestra.run.smithi094.stdout:4144 trunc from 0xc8ce1ee to 0xeab9dea 2024-09-21T22:51:19.350 INFO:teuthology.orchestra.run.smithi094.stdout:4146 writesame 0x53af0e5 thru 0x53bd932 data_size 0x26(0xe84e bytes) 2024-09-21T22:51:19.359 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.359 INFO:teuthology.orchestra.run.smithi094.stdout:4152 write 0x8051002 thru 0x805197e (0x97d bytes) 2024-09-21T22:51:19.365 INFO:teuthology.orchestra.run.smithi094.stdout:4153 read 0x59a0c02 thru 0x59ad7ec (0xcbeb bytes) 2024-09-21T22:51:19.366 INFO:teuthology.orchestra.run.smithi094.stdout:4154 read 0xe8ad0b1 thru 0xe8b733b (0xa28b bytes) 2024-09-21T22:51:19.366 INFO:teuthology.orchestra.run.smithi094.stdout:4156 write 0xfcb37a thru 0xfcd2b5 (0x1f3c bytes) 2024-09-21T22:51:19.385 INFO:teuthology.orchestra.run.smithi094.stdout:4157 write 0x4be8cf thru 0x4c4697 (0x5dc9 bytes) 2024-09-21T22:51:19.404 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:19.396+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:19.422 INFO:teuthology.orchestra.run.smithi094.stdout:4158 trunc from 0xeab9dea to 0x917ddd2 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 34.337356423999999, 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34471 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 34.332181591999998, 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.039490556000000003, 2024-09-21T22:51:19.433 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34471 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093355+0000", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9800000000000004e-07 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093357+0000", 2024-09-21T22:51:19.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5210000000000001e-06 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093365+0000", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093370+0000", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1889999999999998e-06 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128443+0000", 2024-09-21T22:51:19.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035073159999999999 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128522+0000", 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9128000000000005e-05 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.132845+0000", 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043228559999999999 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.937156559000002, 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:19.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:19.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.442 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935642299000001, 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:19.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935588996999996, 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935533949000003, 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.456 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935464764999999, 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:19.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935402197999998, 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.935352803999997, 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:19.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.934849020999998, 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.934781442000002, 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:19.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.934232199, 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.479 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.484 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.934154718000002, 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.485 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:19.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:19.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933990586999997, 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933929608, 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:19.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933885996000001, 2024-09-21T22:51:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:19.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933841514000001, 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933796346000001, 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:19.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.933754049000001, 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:19.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:19.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.448321459999999, 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:19.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:19.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.522 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:19.524 INFO:teuthology.orchestra.run.smithi094.stdout:4159 trunc from 0x917ddd2 to 0x50b3b4e 2024-09-21T22:51:19.524 INFO:teuthology.orchestra.run.smithi094.stdout:4160 punch from 0x36f5b81 to 0x36fc3f5, (0x6874 bytes) 2024-09-21T22:51:19.524 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:19.519+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:19.543 INFO:teuthology.orchestra.run.smithi094.stdout:4162 writesame 0xb888ebf thru 0xb894412 data_size 0x6e(0xb554 bytes) 2024-09-21T22:51:19.544 INFO:teuthology.orchestra.run.smithi094.stdout:4163 write 0x616f94a thru 0x6177999 (0x8050 bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4164 writesame 0xe8ea823 thru 0xe8efe49 data_size 0xb(0x5627 bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4165 read 0x65630cc thru 0x656ba23 (0x8958 bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4166 read 0xcc2c5e3 thru 0xcc2fef8 (0x3916 bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4167 read 0x528acc0 thru 0x528cc77 (0x1fb8 bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4169 read 0xd91d774 thru 0xd921aad (0x433a bytes) 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.571 INFO:teuthology.orchestra.run.smithi094.stdout:4171 punch from 0x431ed55 to 0x4326383, (0x762e bytes) 2024-09-21T22:51:19.589 INFO:teuthology.orchestra.run.smithi094.stdout:4173 trunc from 0xe8efe4a to 0x642818e 2024-09-21T22:51:19.602 INFO:teuthology.orchestra.run.smithi094.stdout:4174 read 0x1d764a4 thru 0x1d81f3f (0xba9c bytes) 2024-09-21T22:51:19.602 INFO:teuthology.orchestra.run.smithi094.stdout:4176 read 0x96df64 thru 0x970dc6 (0x2e63 bytes) 2024-09-21T22:51:19.624 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:19.620+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:19.626 INFO:teuthology.orchestra.run.smithi094.stdout:4177 write 0xa1370f7 thru 0xa1444ea (0xd3f4 bytes) 2024-09-21T22:51:19.637 INFO:teuthology.orchestra.run.smithi094.stdout:4179 write 0x436cd5b thru 0x4373373 (0x6619 bytes) 2024-09-21T22:51:19.649 INFO:teuthology.orchestra.run.smithi094.stdout:4180 write 0x48d7c72 thru 0x48d9749 (0x1ad8 bytes) 2024-09-21T22:51:19.652 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.652 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.652 INFO:teuthology.orchestra.run.smithi094.stdout:4183 writesame 0x2e14908 thru 0x2e22e78 data_size 0x7(0xe571 bytes) 2024-09-21T22:51:19.666 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.666 INFO:teuthology.orchestra.run.smithi094.stdout:4186 read 0x9192b46 thru 0x919f4c3 (0xc97e bytes) 2024-09-21T22:51:19.669 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.669 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:19.669 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:19.669 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:19.690 INFO:teuthology.orchestra.run.smithi094.stdout:4188 write 0xde77b97 thru 0xde7a01b (0x2485 bytes) 2024-09-21T22:51:19.712 INFO:teuthology.orchestra.run.smithi094.stdout:4189 write 0xe7f69ab thru 0xe80402e (0xd684 bytes) 2024-09-21T22:51:19.722 INFO:teuthology.orchestra.run.smithi094.stdout:4190 write 0x2e35e1 thru 0x2f3214 (0xfc34 bytes) 2024-09-21T22:51:19.724 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:19.720+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:19.746 INFO:teuthology.orchestra.run.smithi094.stdout:4191 punch from 0x9fbbbeb to 0x9fc9e00, (0xe215 bytes) 2024-09-21T22:51:19.746 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:19.746 INFO:teuthology.orchestra.run.smithi094.stdout:4193 write 0xc8a2d1a thru 0xc8ac295 (0x957c bytes) 2024-09-21T22:51:19.755 INFO:teuthology.orchestra.run.smithi094.stdout:4195 write 0x28b7684 thru 0x28b8373 (0xcf0 bytes) 2024-09-21T22:51:19.759 INFO:teuthology.orchestra.run.smithi094.stdout:4196 punch from 0x118e63 to 0x12387f, (0xaa1c bytes) 2024-09-21T22:51:19.759 INFO:teuthology.orchestra.run.smithi094.stdout:4197 write 0x23ec16d thru 0x23f371a (0x75ae bytes) 2024-09-21T22:51:19.768 INFO:teuthology.orchestra.run.smithi094.stdout:4198 read 0x17cabd thru 0x181181 (0x46c5 bytes) 2024-09-21T22:51:19.773 INFO:teuthology.orchestra.run.smithi094.stdout:4199 trunc from 0xe80402f to 0xd396001 2024-09-21T22:51:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:4200 read 0xe4c403 thru 0xe522a4 (0x5ea2 bytes) 2024-09-21T22:51:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:4206 write 0xa2068ef thru 0xa2102e7 (0x99f9 bytes) 2024-09-21T22:51:19.813 INFO:teuthology.orchestra.run.smithi094.stdout:4207 trunc from 0xd396001 to 0xa16097d 2024-09-21T22:51:19.814 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.814 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:19.814 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:19.814 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:19.814 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:19.826 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:19.823+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:19.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:19.863 INFO:teuthology.orchestra.run.smithi094.stdout:4210 writesame 0xeb837b2 thru 0xeb8b142 data_size 0x1(0x7991 bytes) 2024-09-21T22:51:19.863 INFO:teuthology.orchestra.run.smithi094.stdout:4211 writesame 0x452b374 thru 0x45376bb data_size 0xc(0xc348 bytes) 2024-09-21T22:51:19.889 INFO:teuthology.orchestra.run.smithi094.stdout:4212 trunc from 0xeb8b143 to 0x4746f59 2024-09-21T22:51:19.926 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:19.923+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:4213 writesame 0xc6b84aa thru 0xc6b98a2 data_size 0x1(0x13f9 bytes) 2024-09-21T22:51:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:4214 read 0x9b5c602 thru 0x9b6b47c (0xee7b bytes) 2024-09-21T22:51:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:4215 write 0xc3839c3 thru 0xc388dd3 (0x5411 bytes) 2024-09-21T22:51:19.964 INFO:teuthology.orchestra.run.smithi094.stdout:4217 writesame 0x971cbb2 thru 0x9725b0b data_size 0x2(0x8f5a bytes) 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.755923621999997, 2024-09-21T22:51:19.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:19.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:13689 3.4 e46/24 3:264d5c02:::rbd_data.111e4c6c7d62.00000000000002b9:head v 46'113, mlcod=46'111)", 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:37.236307+0000", 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.746843480999999, 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028733029999999998, 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236307+0000", 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236307+0000", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236309+0000", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9800000000000001e-06 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236353+0000", 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3859999999999997e-05 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236390+0000", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7178999999999998e-05 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236424+0000", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4023999999999998e-05 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236575+0000", 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015076900000000001 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.236612+0000", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7082999999999997e-05 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.239055+0000", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024431259999999999 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:37.239180+0000", 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125282 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.939360624000003, 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:19.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.762395486000003, 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:19.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:19.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:20.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:20.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36671 3.0 e53/24 3:03a43115:::rbd_data.111e4aac08c7.00000000000001e6:head v 53'401, mlcod=53'397)", 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.740578030999998, 2024-09-21T22:51:20.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003075979, 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242575+0000", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5670000000000002e-06 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242594+0000", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8448999999999999e-05 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242636+0000", 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2756999999999997e-05 2024-09-21T22:51:20.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242656+0000", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9643000000000001e-05 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242810+0000", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000153523 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242868+0000", 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8582000000000003e-05 2024-09-21T22:51:20.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245513+0000", 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026443489999999998 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245649+0000", 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013610900000000001 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 30.15128683, 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:20.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.971481631, 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:20.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:20.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.893861959999999, 2024-09-21T22:51:20.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:20.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.893596688999999, 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:20.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.563523891999999, 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:20.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:20.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.382437941999999, 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:20.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:20.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.367822576000002, 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:20.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.367439985000001, 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.037 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:20.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:20.041 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.365427521000001, 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:20.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.336702721000002, 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:20.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:20.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:20.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.334009950999999, 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:20.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:107624 3.9 e70/24 3:9604045b:::rbd_data.111eaf8fb188.0000000000000009:head v 70'215, mlcod=68'213)", 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.9815760579999999, 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029782390000000001, 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1300000000000001e-07 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002740+0000", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001164691 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002751+0000", 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0978000000000001e-05 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002759+0000", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1820000000000006e-06 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002838+0000", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.962e-05 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002868+0000", 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9691000000000001e-05 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004502+0000", 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001633858 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004553+0000", 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0905999999999997e-05 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:124116 3.0 e74/24 3:00284fc4:::rbd_data.111e6870f364.00000000000002be:head v 74'484, mlcod=73'480)", 2024-09-21T22:51:20.058 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:14.284185+0000", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.6989655409999997, 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028433999999999998, 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284185+0000", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284185+0000", 2024-09-21T22:51:20.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284187+0000", 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6080000000000001e-06 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284198+0000", 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1240000000000001e-05 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284223+0000", 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5301999999999999e-05 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284234+0000", 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0271999999999999e-05 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284325+0000", 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1688999999999996e-05 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.284346+0000", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0947000000000001e-05 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.286979+0000", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026327210000000002 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:14.287029+0000", 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9620999999999998e-05 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147260 3.4 e79/24 3:26d7bf4e:::rbd_data.111e2790987.0000000000000004:head v 79'490, mlcod=74'488)", 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.57417909600000006, 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030032000000000001, 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:20.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409517+0000", 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000545047 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409532+0000", 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4669e-05 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409540+0000", 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6670000000000002e-06 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409607+0000", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6264000000000001e-05 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409636+0000", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9166000000000002e-05 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411936+0000", 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022996959999999999 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411975+0000", 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9251000000000001e-05 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.57355032699999997, 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:20.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:20.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:20.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:20.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:20.071 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:20.072 INFO:teuthology.orchestra.run.smithi094.stdout:4219 punch from 0x125e14 to 0x135dd1, (0xffbd bytes) 2024-09-21T22:51:20.072 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.072 INFO:teuthology.orchestra.run.smithi094.stdout:4222 write 0x28755f2 thru 0x2877b99 (0x25a8 bytes) 2024-09-21T22:51:20.072 INFO:teuthology.orchestra.run.smithi094.stdout:4223 read 0x19d79b8 thru 0x19ddc85 (0x62ce bytes) 2024-09-21T22:51:20.072 INFO:teuthology.orchestra.run.smithi094.stdout:4224 read 0xac079b2 thru 0xac17151 (0xf7a0 bytes) 2024-09-21T22:51:20.073 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.073 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.073 INFO:teuthology.orchestra.run.smithi094.stdout:4227 write 0xb6b6acd thru 0xb6c4d01 (0xe235 bytes) 2024-09-21T22:51:20.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:20.075 INFO:teuthology.orchestra.run.smithi094.stdout:4231 read 0xb81493e thru 0xb815a18 (0x10db bytes) 2024-09-21T22:51:20.075 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4233 write 0xb5ca5b thru 0xb6847c (0xba22 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4234 read 0x140b701 thru 0x140c936 (0x1236 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4235 trunc from 0xc6b98a3 to 0xc67fc5a 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4236 read 0xa0a017 thru 0xa1117b (0x7165 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4237 writesame 0xdba857c thru 0xdbb6e86 data_size 0x1(0xe90b bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4239 read 0x68ebd5a thru 0x68ec9bb (0xc62 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4240 read 0xd3364eb thru 0xd33f83c (0x9352 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4241 write 0x41f9231 thru 0x41fce2e (0x3bfe bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4242 write 0x281c247 thru 0x2829d7e (0xdb38 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4243 punch from 0xac3e69b to 0xac49c38, (0xb59d bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:4244 punch from 0xb572417 to 0xb579860, (0x7449 bytes) 2024-09-21T22:51:20.076 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.079 INFO:teuthology.orchestra.run.smithi094.stdout:4246 trunc from 0xdbb6e87 to 0x53d11a5 2024-09-21T22:51:20.080 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:20.076+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:20.106 INFO:teuthology.orchestra.run.smithi094.stdout:4247 trunc from 0x53d11a5 to 0x550de8 2024-09-21T22:51:20.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:20.188 INFO:teuthology.orchestra.run.smithi094.stdout:4248 write 0xd4edaed thru 0xd4f53bf (0x78d3 bytes) 2024-09-21T22:51:20.188 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:20.184+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:20.194 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.194 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.194 INFO:teuthology.orchestra.run.smithi094.stdout:4251 writesame 0x9d166c0 thru 0x9d16d55 data_size 0x3(0x696 bytes) 2024-09-21T22:51:20.199 INFO:teuthology.orchestra.run.smithi094.stdout:4252 write 0x7857d2b thru 0x7862403 (0xa6d9 bytes) 2024-09-21T22:51:20.205 INFO:teuthology.orchestra.run.smithi094.stdout:4253 write 0x73e5379 thru 0x73f450d (0xf195 bytes) 2024-09-21T22:51:20.211 INFO:teuthology.orchestra.run.smithi094.stdout:4254 write 0x993afc3 thru 0x9944a9f (0x9add bytes) 2024-09-21T22:51:20.217 INFO:teuthology.orchestra.run.smithi094.stdout:4256 write 0xc6e137e thru 0xc6eb828 (0xa4ab bytes) 2024-09-21T22:51:20.223 INFO:teuthology.orchestra.run.smithi094.stdout:4260 read 0x37a43a5 thru 0x37b1237 (0xce93 bytes) 2024-09-21T22:51:20.223 INFO:teuthology.orchestra.run.smithi094.stdout:4261 write 0x27b2680 thru 0x27b7373 (0x4cf4 bytes) 2024-09-21T22:51:20.230 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:20.230 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:20.230 INFO:teuthology.orchestra.run.smithi094.stdout:4262 trunc from 0xd4f53c0 to 0xbaa0cc7 2024-09-21T22:51:20.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:20.247 INFO:teuthology.orchestra.run.smithi094.stdout:4263 read 0x47fbf9c thru 0x480067f (0x46e4 bytes) 2024-09-21T22:51:20.247 INFO:teuthology.orchestra.run.smithi094.stdout:4265 punch from 0xa2e094c to 0xa2e3837, (0x2eeb bytes) 2024-09-21T22:51:20.247 INFO:teuthology.orchestra.run.smithi094.stdout:4266 write 0xb39bb9f thru 0xb39be4f (0x2b1 bytes) 2024-09-21T22:51:20.261 INFO:teuthology.orchestra.run.smithi094.stdout:4267 write 0xc4cd31c thru 0xc4dcea4 (0xfb89 bytes) 2024-09-21T22:51:20.269 INFO:teuthology.orchestra.run.smithi094.stdout:4269 writesame 0x327fefb thru 0x32883c2 data_size 0x1c(0x84c8 bytes) 2024-09-21T22:51:20.277 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:20.272+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4270 read 0x2b64be7 thru 0x2b67521 (0x293b bytes) 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4271 read 0x3066290 thru 0x3071e2c (0xbb9d bytes) 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4272 punch from 0x66e3684 to 0x66ee24f, (0xabcb bytes) 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4273 read 0x4c04b88 thru 0x4c07210 (0x2689 bytes) 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4277 punch from 0x94df6f to 0x95b528, (0xd5b9 bytes) 2024-09-21T22:51:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:4278 write 0x22c94ce thru 0x22cdd13 (0x4846 bytes) 2024-09-21T22:51:20.282 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.283 INFO:teuthology.orchestra.run.smithi094.stdout:4280 write 0x4b31838 thru 0x4b3ac8a (0x9453 bytes) 2024-09-21T22:51:20.287 INFO:teuthology.orchestra.run.smithi094.stdout:4282 writesame 0x340229a thru 0x34055e7 data_size 0x3(0x334e bytes) 2024-09-21T22:51:20.293 INFO:teuthology.orchestra.run.smithi094.stdout:4283 read 0x2929ad8 thru 0x2935590 (0xbab9 bytes) 2024-09-21T22:51:20.294 INFO:teuthology.orchestra.run.smithi094.stdout:4284 read 0x3763a88 thru 0x376f84b (0xbdc4 bytes) 2024-09-21T22:51:20.294 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.294 INFO:teuthology.orchestra.run.smithi094.stdout:4286 writesame 0x34f0cab thru 0x34fb3e4 data_size 0xf(0xa73a bytes) 2024-09-21T22:51:20.299 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.299 INFO:teuthology.orchestra.run.smithi094.stdout:4290 write 0x30624f7 thru 0x306df73 (0xba7d bytes) 2024-09-21T22:51:20.302 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:20.302 INFO:teuthology.orchestra.run.smithi094.stdout:4291 read 0x565cfa0 thru 0x565fed2 (0x2f33 bytes) 2024-09-21T22:51:20.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:51:20.313 INFO:teuthology.orchestra.run.smithi094.stdout:4292 write 0xe048e8e thru 0xe053368 (0xa4db bytes) 2024-09-21T22:51:20.320 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:20.321 INFO:teuthology.orchestra.run.smithi094.stdout:4293 read 0xad2a2e7 thru 0xad2c981 (0x269b bytes) 2024-09-21T22:51:20.321 INFO:teuthology.orchestra.run.smithi094.stdout:4294 read 0x3e1ce21 thru 0x3e29d8e (0xcf6e bytes) 2024-09-21T22:51:20.321 INFO:teuthology.orchestra.run.smithi094.stdout:4295 write 0xb4f4f23 thru 0xb5029b9 (0xda97 bytes) 2024-09-21T22:51:20.326 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.326 INFO:teuthology.orchestra.run.smithi094.stdout:4297 read 0x9943796 thru 0x9946025 (0x2890 bytes) 2024-09-21T22:51:20.326 INFO:teuthology.orchestra.run.smithi094.stdout:4298 write 0x43c5b08 thru 0x43cc2e1 (0x67da bytes) 2024-09-21T22:51:20.331 INFO:teuthology.orchestra.run.smithi094.stdout:4299 read 0x65533f8 thru 0x65592be (0x5ec7 bytes) 2024-09-21T22:51:20.331 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.331 INFO:teuthology.orchestra.run.smithi094.stdout:4303 write 0xc43487e thru 0xc442fc0 (0xe743 bytes) 2024-09-21T22:51:20.332 INFO:teuthology.orchestra.run.smithi094.stdout:4304 punch from 0xb131364 to 0xb1321da, (0xe76 bytes) 2024-09-21T22:51:20.332 INFO:teuthology.orchestra.run.smithi094.stdout:4305 writesame 0x9419aca thru 0x9427265 data_size 0x2(0xd79c bytes) 2024-09-21T22:51:20.340 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:20.341 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:20.347 INFO:teuthology.orchestra.run.smithi094.stdout:4306 write 0x6fecf47 thru 0x6ff73a4 (0xa45e bytes) 2024-09-21T22:51:20.352 INFO:teuthology.orchestra.run.smithi094.stdout:4308 writesame 0xa612399 thru 0xa6211bf data_size 0x1(0xee27 bytes) 2024-09-21T22:51:20.366 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:20.366 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:20.374 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:20.378 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:20.372+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:20.386 INFO:teuthology.orchestra.run.smithi094.stdout:4309 punch from 0x2ed4f75 to 0x2edfe7b, (0xaf06 bytes) 2024-09-21T22:51:20.386 INFO:teuthology.orchestra.run.smithi094.stdout:4312 punch from 0x764df94 to 0x7650c2a, (0x2c96 bytes) 2024-09-21T22:51:20.386 INFO:teuthology.orchestra.run.smithi094.stdout:4313 read 0xbee4e39 thru 0xbef3e85 (0xf04d bytes) 2024-09-21T22:51:20.386 INFO:teuthology.orchestra.run.smithi094.stdout:4314 write 0x1047ffe thru 0x1049444 (0x1447 bytes) 2024-09-21T22:51:20.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:20.388 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:20.389 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:20.391 INFO:teuthology.orchestra.run.smithi094.stdout:4315 write 0x624b69e thru 0x6251130 (0x5a93 bytes) 2024-09-21T22:51:20.406 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:20.410 INFO:teuthology.orchestra.run.smithi094.stdout:4316 trunc from 0xe053369 to 0x508d8d7 2024-09-21T22:51:20.428 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:20.428 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:4318 read 0x19cb6c2 thru 0x19d91ab (0xdaea bytes) 2024-09-21T22:51:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:4320 punch from 0x5ec4ba to 0x5f72d4, (0xae1a bytes) 2024-09-21T22:51:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:4323 punch from 0x3e737e1 to 0x3e80016, (0xc835 bytes) 2024-09-21T22:51:20.432 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:20.433 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:20.436 INFO:teuthology.orchestra.run.smithi094.stdout:4325 trunc from 0x508d8d7 to 0x8bc6f9 2024-09-21T22:51:20.454 INFO:teuthology.orchestra.run.smithi094.stdout:4326 read 0x6f6da9 thru 0x704a43 (0xdc9b bytes) 2024-09-21T22:51:20.454 INFO:teuthology.orchestra.run.smithi094.stdout:4327 read 0x6e878f thru 0x6f2dce (0xa640 bytes) 2024-09-21T22:51:20.478 INFO:teuthology.orchestra.run.smithi094.stdout:4328 writesame 0xdcce2ce thru 0xdcd6b2f data_size 0x45(0x8862 bytes) 2024-09-21T22:51:20.478 INFO:teuthology.orchestra.run.smithi094.stdout:4329 writesame 0xd51d498 thru 0xd5276c4 data_size 0x15(0xa22d bytes) 2024-09-21T22:51:20.478 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:20.472+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:20.487 INFO:teuthology.orchestra.run.smithi094.stdout:4333 read 0x5d058c thru 0x5d2cf1 (0x2766 bytes) 2024-09-21T22:51:20.498 INFO:teuthology.orchestra.run.smithi094.stdout:4334 trunc from 0xdcd6b30 to 0x53a120f 2024-09-21T22:51:20.516 INFO:teuthology.orchestra.run.smithi094.stdout:4335 read 0x3fe791a thru 0x3feb00d (0x36f4 bytes) 2024-09-21T22:51:20.516 INFO:teuthology.orchestra.run.smithi094.stdout:4336 read 0x366b391 thru 0x3679d50 (0xe9c0 bytes) 2024-09-21T22:51:20.516 INFO:teuthology.orchestra.run.smithi094.stdout:4337 read 0x3c97090 thru 0x3ca2875 (0xb7e6 bytes) 2024-09-21T22:51:20.516 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:4340 punch from 0x3aba7d3 to 0x3abce99, (0x26c6 bytes) 2024-09-21T22:51:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:4341 read 0x4b49f42 thru 0x4b58245 (0xe304 bytes) 2024-09-21T22:51:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:4345 read 0x32b9fe9 thru 0x32c9cee (0xfd06 bytes) 2024-09-21T22:51:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:4346 write 0x31b74cd thru 0x31bd9ff (0x6533 bytes) 2024-09-21T22:51:20.525 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:20.525 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.c 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.7 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.2 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.1f 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.8 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.1e 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.b 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.5 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.18 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.15 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:3.19 2024-09-21T22:51:20.526 INFO:teuthology.orchestra.run.smithi094.stdout:2.1 2024-09-21T22:51:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:4347 trunc from 0x53a120f to 0xe5f798c 2024-09-21T22:51:20.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:4348 clone 27 order 23 su 32768 sc 7 2024-09-21T22:51:20.585 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:20.583+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:20.662 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:20.662 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:20.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:20.718 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:20.715+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:20.782 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:20.776+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:20.839 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:20.839 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:20.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:20.897 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:20.895+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:20.977 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:20.977 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:20.984 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:20.980+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:20.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:21.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.15 --file /var/log/ceph/exp.3.15.3 2024-09-21T22:51:21.124 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:21.120+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:21.155 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:21.155 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:21.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:21.185 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:21.180+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:21.285 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:21.283+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:21.293 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:21.293 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:21.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:21.312 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:51:21.398 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:21.395+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:21.480 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:21.480 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:21.480 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone26 from 0xe5f798c (overlap 0x550de8) to 0x0 2024-09-21T22:51:21.488 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:21.480+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:21.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:21.593 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:21.591+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:21.621 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T22:51:21.624 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:21.624 INFO:teuthology.orchestra.run.smithi094.stdout:4350 read 0x4704062 thru 0x4708993 (0x4932 bytes) 2024-09-21T22:51:21.625 INFO:teuthology.orchestra.run.smithi094.stdout:4351 writesame 0xd221907 thru 0xd227d8e data_size 0x4(0x6488 bytes) 2024-09-21T22:51:21.648 INFO:teuthology.orchestra.run.smithi094.stdout:4352 write 0x8a0a149 thru 0x8a0e72c (0x45e4 bytes) 2024-09-21T22:51:21.657 INFO:teuthology.orchestra.run.smithi094.stdout:4353 write 0x8070377 thru 0x807a8d2 (0xa55c bytes) 2024-09-21T22:51:21.668 INFO:teuthology.orchestra.run.smithi094.stdout:4355 write 0x36d1563 thru 0x36db4f2 (0x9f90 bytes) 2024-09-21T22:51:21.679 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:21.679 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:21.690 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:21.684+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:21.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:21.703 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:21.703 INFO:teuthology.orchestra.run.smithi094.stdout:4358 write 0x95db590 thru 0x95e61ce (0xac3f bytes) 2024-09-21T22:51:21.708 INFO:teuthology.orchestra.run.smithi094.stdout:4359 writesame 0x1e2987e thru 0x1e2f8a9 data_size 0x5(0x602c bytes) 2024-09-21T22:51:21.736 INFO:teuthology.orchestra.run.smithi094.stdout:4360 punch from 0xc413b2 to 0xc49316, (0x7f64 bytes) 2024-09-21T22:51:21.736 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:21.736 INFO:teuthology.orchestra.run.smithi094.stdout:4362 read 0x25189bf thru 0x2525824 (0xce66 bytes) 2024-09-21T22:51:21.749 INFO:teuthology.orchestra.run.smithi094.stdout:4363 writesame 0xec6ee06 thru 0xec705d5 data_size 0x10(0x17d0 bytes) 2024-09-21T22:51:21.758 INFO:teuthology.orchestra.run.smithi094.stdout:4364 writesame 0x8db8a2e thru 0x8dbf4c6 data_size 0x1(0x6a99 bytes) 2024-09-21T22:51:21.777 INFO:teuthology.orchestra.run.smithi094.stdout:4365 punch from 0xc174161 to 0xc17d49f, (0x933e bytes) 2024-09-21T22:51:21.777 INFO:teuthology.orchestra.run.smithi094.stdout:4367 punch from 0x9b475ed to 0x9b47c90, (0x6a3 bytes) 2024-09-21T22:51:21.785 INFO:teuthology.orchestra.run.smithi094.stdout:4368 trunc from 0xec705d6 to 0xb4f2d22 2024-09-21T22:51:21.791 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:21.784+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:21.811 INFO:teuthology.orchestra.run.smithi094.stdout:4369 read 0x553baa8 thru 0x5544fa1 (0x94fa bytes) 2024-09-21T22:51:21.824 INFO:teuthology.orchestra.run.smithi094.stdout:4370 writesame 0xd3ac264 thru 0xd3badc0 data_size 0x1(0xeb5d bytes) 2024-09-21T22:51:21.838 INFO:teuthology.orchestra.run.smithi094.stdout:4372 trunc from 0xd3badc1 to 0x5bf985a 2024-09-21T22:51:21.870 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:21.870 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:21.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:21.890 INFO:teuthology.orchestra.run.smithi094.stdout:4373 writesame 0x640d493 thru 0x64110a2 data_size 0x8(0x3c10 bytes) 2024-09-21T22:51:21.893 INFO:teuthology.orchestra.run.smithi094.stdout:4374 read 0x1411127 thru 0x1419be2 (0x8abc bytes) 2024-09-21T22:51:21.894 INFO:teuthology.orchestra.run.smithi094.stdout:4376 read 0x2855fe4 thru 0x2865921 (0xf93e bytes) 2024-09-21T22:51:21.894 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:21.894 INFO:teuthology.orchestra.run.smithi094.stdout:4379 punch from 0x4d600d9 to 0x4d6016f, (0x96 bytes) 2024-09-21T22:51:21.894 INFO:teuthology.orchestra.run.smithi094.stdout:4380 punch from 0x29b9b77 to 0x29bb66e, (0x1af7 bytes) 2024-09-21T22:51:21.909 INFO:teuthology.orchestra.run.smithi094.stdout:4381 trunc from 0x64110a3 to 0xd5d66a7 2024-09-21T22:51:21.918 INFO:teuthology.orchestra.run.smithi094.stdout:4383 read 0x202eada thru 0x2039beb (0xb112 bytes) 2024-09-21T22:51:21.918 INFO:teuthology.orchestra.run.smithi094.stdout:4386 read 0x5c03ec thru 0x5c62db (0x5ef0 bytes) 2024-09-21T22:51:21.919 INFO:teuthology.orchestra.run.smithi094.stdout:4387 read 0xa710b85 thru 0xa71d24e (0xc6ca bytes) 2024-09-21T22:51:21.919 INFO:teuthology.orchestra.run.smithi094.stdout:4388 read 0x8f3b313 thru 0x8f3eda6 (0x3a94 bytes) 2024-09-21T22:51:21.919 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:21.919 INFO:teuthology.orchestra.run.smithi094.stdout:4390 punch from 0xac0f8a1 to 0xac1a358, (0xaab7 bytes) 2024-09-21T22:51:21.919 INFO:teuthology.orchestra.run.smithi094.stdout:4392 write 0x4cc2b1 thru 0x4d6b9f (0xa8ef bytes) 2024-09-21T22:51:21.929 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:21.927+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:21.940 INFO:teuthology.orchestra.run.smithi094.stdout:4394 read 0x27dedd2 thru 0x27e4945 (0x5b74 bytes) 2024-09-21T22:51:21.941 INFO:teuthology.orchestra.run.smithi094.stdout:4395 writesame 0x90ccb2a thru 0x90d08d9 data_size 0x18(0x3db0 bytes) 2024-09-21T22:51:21.970 INFO:teuthology.orchestra.run.smithi094.stdout:4396 trunc from 0xd5d66a7 to 0x3745c9d 2024-09-21T22:51:21.993 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:21.987+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:22.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:22.033 INFO:teuthology.orchestra.run.smithi094.stdout:4397 write 0x314bc7e thru 0x3154439 (0x87bc bytes) 2024-09-21T22:51:22.036 INFO:teuthology.orchestra.run.smithi094.stdout:4398 read 0x176e2cc thru 0x176fb1a (0x184f bytes) 2024-09-21T22:51:22.036 INFO:teuthology.orchestra.run.smithi094.stdout:4401 clone 28 order 25 su 131072 sc 3 2024-09-21T22:51:22.093 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:22.087+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:22.156 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:22.156 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:22.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:22.193 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:22.187+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:22.294 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:22.291+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:22.340 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:22.340 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:22.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:22.400 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:22.396+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:22.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:22.480 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:22.480 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:22.495 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:22.488+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:22.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:22.510 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone27 from 0x3745c9d (overlap 0x3745c9d) to 0x2c37d4a 2024-09-21T22:51:22.589 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:22.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:22.597 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:22.591+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:22.606 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:22.624 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:22.625 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:22.625 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:22.636 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:22.636 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:22.636 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:22.636 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:22.646 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:22.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:22.669 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:22.669 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:Exporting 3.15 info 3.15( v 74'308 (0'0,74'308] local-lis/les=24/25 n=70 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) 2024-09-21T22:51:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8037206:::rbd_data.111e6870f364.0000000000000075:head# 2024-09-21T22:51:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a818c062:::rbd_data.111e6870f364.0000000000000062:head# 2024-09-21T22:51:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a82b3153:::rbd_data.111eaadcd46c.00000000000002bd:head# 2024-09-21T22:51:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8314351:::rbd_data.111e7572a6e.0000000000000015:head# 2024-09-21T22:51:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a842e28b:::rbd_data.111e7572a6e.000000000000002a:head# 2024-09-21T22:51:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8a5464f:::rbd_data.111e38708da6.0000000000000095:f# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8a5464f:::rbd_data.111e38708da6.0000000000000095:head# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8c34c3c:::rbd_data.111e38708da6.0000000000000069:head# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a8e5cbc8:::rbd_data.111eaadcd46c.0000000000000015:head# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a905ddda:::rbd_data.111e4aac08c7.0000000000000331:10# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a905ddda:::rbd_data.111e4aac08c7.0000000000000331:head# 2024-09-21T22:51:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a92c4996:::rbd_data.111e7572a6e.0000000000000071:6# 2024-09-21T22:51:22.696 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a92c4996:::rbd_data.111e7572a6e.0000000000000071:head# 2024-09-21T22:51:22.696 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a94a38c9:::rbd_data.111e33663d62.000000000000006a:head# 2024-09-21T22:51:22.696 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a952b6c7:::rbd_data.111e9d7ae6fb.000000000000005c:head# 2024-09-21T22:51:22.696 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a95575fd:::rbd_data.111e7572a6e.000000000000002e:head# 2024-09-21T22:51:22.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a98d08ab:::rbd_data.111e4c6c7d62.0000000000000068:head# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a9f62cbd:::rbd_data.111e9d7ae6fb.0000000000000091:head# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:a9fb508b:::rbd_object_map.111eaadcd46c.0000000000000018:head# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:4# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:head# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aabc28af:::rbd_data.111ee1b766.0000000000000005:14# 2024-09-21T22:51:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aabc28af:::rbd_data.111ee1b766.0000000000000005:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aadc9c5f:::rbd_data.111e38708da6.0000000000000046:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab1b671d:::rbd_data.111e7572a6e.0000000000000061:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab3b4851:::rbd_data.111e33663d62.0000000000000061:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab3d6083:::rbd_data.111e4aac08c7.000000000000002c:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab593a5c:::rbd_data.111e9d7ae6fb.0000000000000077:head# 2024-09-21T22:51:22.700 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab998f4c:::rbd_data.111e38708da6.000000000000021a:f# 2024-09-21T22:51:22.701 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:22.695+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ab998f4c:::rbd_data.111e38708da6.000000000000021a:head# 2024-09-21T22:51:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ac18174b:::rbd_data.111e4c6c7d62.0000000000000008:d# 2024-09-21T22:51:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ac18174b:::rbd_data.111e4c6c7d62.0000000000000008:head# 2024-09-21T22:51:22.702 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ac275cda:::rbd_header.111e9d7ae6fb:head# 2024-09-21T22:51:22.702 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ac4d23f4:::rbd_data.111e38708da6.00000000000000ab:head# 2024-09-21T22:51:22.702 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ac7d6307:::rbd_data.111e752d4c95.0000000000000001:head# 2024-09-21T22:51:22.707 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:acc37f03:::rbd_data.111eaadcd46c.00000000000002df:head# 2024-09-21T22:51:22.707 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17# 2024-09-21T22:51:22.716 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:head# 2024-09-21T22:51:22.716 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad189c28:::rbd_data.111e7572a6e.0000000000000046:head# 2024-09-21T22:51:22.716 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad246106:::rbd_data.111e38708da6.0000000000000023:f# 2024-09-21T22:51:22.716 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad246106:::rbd_data.111e38708da6.0000000000000023:head# 2024-09-21T22:51:22.716 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad53f7c1:::rbd_data.111eaadcd46c.000000000000017f:head# 2024-09-21T22:51:22.717 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad659c07:::rbd_data.111eeb79a41d.000000000000006d:head# 2024-09-21T22:51:22.717 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad6dc33b:::rbd_data.111e4aac08c7.000000000000007a:10# 2024-09-21T22:51:22.717 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad6dc33b:::rbd_data.111e4aac08c7.000000000000007a:head# 2024-09-21T22:51:22.718 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad836215:::rbd_data.111e5a344aa.0000000000000032:13# 2024-09-21T22:51:22.718 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T22:51:22.718 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad836215:::rbd_data.111e5a344aa.0000000000000032:head# 2024-09-21T22:51:22.719 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad8798ad:::rbd_data.111e4c6c7d62.000000000000006f:head# 2024-09-21T22:51:22.719 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:22.719 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ad97a3a2:::rbd_data.111e38708da6.0000000000000064:head# 2024-09-21T22:51:22.719 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:22.720 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ada6a5f3:::rbd_data.111e9d7ae6fb.0000000000000055:head# 2024-09-21T22:51:22.720 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17# 2024-09-21T22:51:22.720 INFO:teuthology.orchestra.run.smithi094.stdout:4402 read 0x365257f thru 0x365f079 (0xcafb bytes) 2024-09-21T22:51:22.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:22.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:22.723 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:22.723 INFO:teuthology.orchestra.run.smithi094.stdout:4404 writesame 0x9a6452 thru 0x9abb7f data_size 0x1(0x572e bytes) 2024-09-21T22:51:22.727 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:head# 2024-09-21T22:51:22.727 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ae95d2ce:::rbd_data.111ee1b766.0000000000000003:14# 2024-09-21T22:51:22.727 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:ae95d2ce:::rbd_data.111ee1b766.0000000000000003:head# 2024-09-21T22:51:22.728 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aeb2ffa0:::rbd_data.111e4c6c7d62.000000000000008b:head# 2024-09-21T22:51:22.728 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aeb4c717:::rbd_data.111e9d7ae6fb.000000000000003d:head# 2024-09-21T22:51:22.728 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aec00675:::rbd_data.111ea6ec99f1.0000000000000197:5# 2024-09-21T22:51:22.729 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aec00675:::rbd_data.111ea6ec99f1.0000000000000197:head# 2024-09-21T22:51:22.729 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aec4f8d7:::rbd_data.111e20a3ef21.0000000000000119:7# 2024-09-21T22:51:22.730 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:aec4f8d7:::rbd_data.111e20a3ef21.0000000000000119:head# 2024-09-21T22:51:22.730 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af17b25c:::rbd_data.111e33663d62.000000000000006d:head# 2024-09-21T22:51:22.730 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af1c842d:::rbd_data.111ee1b766.000000000000013e:head# 2024-09-21T22:51:22.730 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af3e41ae:::rbd_data.111e9d7ae6fb.0000000000000058:head# 2024-09-21T22:51:22.731 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b# 2024-09-21T22:51:22.731 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af5dc254:::rbd_data.111e6870f364.0000000000000307:head# 2024-09-21T22:51:22.731 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af6d7613:::rbd_data.111edc482e6c.0000000000000000:head# 2024-09-21T22:51:22.745 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b# 2024-09-21T22:51:22.746 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:head# 2024-09-21T22:51:22.746 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:afce5e93:::rbd_data.111eaadcd46c.00000000000002bb:head# 2024-09-21T22:51:22.746 INFO:teuthology.orchestra.run.smithi094.stderr:Read #3:afde4ef8:::rbd_data.111e33663d62.0000000000000014:head# 2024-09-21T22:51:22.746 INFO:teuthology.orchestra.run.smithi094.stderr:Export successful 2024-09-21T22:51:22.801 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:22.795+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:22.802 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:22.802 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:22.803 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:22.803 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:22.803 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:22.817 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:22.904 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:22.900+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:4405 read 0x194ea33 thru 0x1959c8c (0xb25a bytes) 2024-09-21T22:51:22.953 INFO:teuthology.orchestra.run.smithi094.stdout:4407 read 0x189ce4c thru 0x18ab331 (0xe4e6 bytes) 2024-09-21T22:51:22.954 INFO:teuthology.orchestra.run.smithi094.stdout:4408 read 0x20cb51e thru 0x20d814c (0xcc2f bytes) 2024-09-21T22:51:22.955 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.886790161999997, 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:22.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34471 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.881615330000002, 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.039490556000000003, 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:22.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34471 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093354+0000", 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:22.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093355+0000", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9800000000000004e-07 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093357+0000", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5210000000000001e-06 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093365+0000", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.093370+0000", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1889999999999998e-06 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128443+0000", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035073159999999999 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128522+0000", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9128000000000005e-05 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:22.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.132845+0000", 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043228559999999999 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.486590296999999, 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:22.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:22.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:22.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.485076036999999, 2024-09-21T22:51:22.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:22.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:22.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.485022735000001, 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:22.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:22.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.484967687000001, 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:23.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:23.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.484898502999997, 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:23.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.484835936000003, 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:23.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:23.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.484786542000002, 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:23.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.484282759000003, 2024-09-21T22:51:23.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:23.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.48421518, 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.022 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483665936999998, 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483588456, 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:23.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483424325000001, 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:23.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483363345999997, 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:23.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:23.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483319733999998, 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483275251999999, 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:23.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:23.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483230083999999, 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.483187786999999, 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:23.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:23.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:23.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.997755198, 2024-09-21T22:51:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.067 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:23.068 INFO:teuthology.orchestra.run.smithi094.stdout:4411 writesame 0xba6d6bf thru 0xba72890 data_size 0x3(0x51d2 bytes) 2024-09-21T22:51:23.068 INFO:teuthology.orchestra.run.smithi094.stdout:4413 write 0x422fdc9 thru 0x423eb9c (0xedd4 bytes) 2024-09-21T22:51:23.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:23.069 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:23.063+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:23.170 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:23.167+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:23.216 INFO:teuthology.orchestra.run.smithi094.stdout:4414 read 0x1fbad56 thru 0x1fbae30 (0xdb bytes) 2024-09-21T22:51:23.217 INFO:teuthology.orchestra.run.smithi094.stdout:4415 write 0x305407 thru 0x31243a (0xd034 bytes) 2024-09-21T22:51:23.255 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:23.255 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:23.255 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:23.255 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:23.270 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:23.264+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:23.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:23.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.15 2024-09-21T22:51:23.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:51:23.377 INFO:teuthology.orchestra.run.smithi094.stdout:4417 writesame 0x401a4a4 thru 0x40274c5 data_size 0x1(0xd022 bytes) 2024-09-21T22:51:23.378 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:23.372+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:23.403 INFO:teuthology.orchestra.run.smithi094.stdout:4418 read 0x1b4ce2e thru 0x1b52446 (0x5619 bytes) 2024-09-21T22:51:23.404 INFO:teuthology.orchestra.run.smithi094.stdout:4419 write 0xb0bd950 thru 0xb0c8cd8 (0xb389 bytes) 2024-09-21T22:51:23.414 INFO:teuthology.orchestra.run.smithi094.stdout:4420 write 0x8684c30 thru 0x868dbce (0x8f9f bytes) 2024-09-21T22:51:23.422 INFO:teuthology.orchestra.run.smithi094.stdout:4421 punch from 0x283f5bd to 0x284275b, (0x319e bytes) 2024-09-21T22:51:23.422 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.422 INFO:teuthology.orchestra.run.smithi094.stdout:4423 write 0x8c0ad24 thru 0x8c153c9 (0xa6a6 bytes) 2024-09-21T22:51:23.442 INFO:teuthology.orchestra.run.smithi094.stdout:4424 write 0xe71cf70 thru 0xe71f84b (0x28dc bytes) 2024-09-21T22:51:23.449 INFO:teuthology.orchestra.run.smithi094.stdout:4425 writesame 0xc467b33 thru 0xc46db9c data_size 0x2b(0x606a bytes) 2024-09-21T22:51:23.455 INFO:teuthology.orchestra.run.smithi094.stdout:4428 write 0x6cd79be thru 0x6cdbb28 (0x416b bytes) 2024-09-21T22:51:23.459 INFO:teuthology.orchestra.run.smithi094.stdout:4429 write 0x5d684b1 thru 0x5d6b54e (0x309e bytes) 2024-09-21T22:51:23.461 INFO:teuthology.orchestra.run.smithi094.stdout:4430 writesame 0xe3470f0 thru 0xe34f5c7 data_size 0x34(0x84d8 bytes) 2024-09-21T22:51:23.465 INFO:teuthology.orchestra.run.smithi094.stdout:4433 write 0xb0a11f3 thru 0xb0a966c (0x847a bytes) 2024-09-21T22:51:23.468 INFO:teuthology.orchestra.run.smithi094.stdout:4434 write 0xb43af9e thru 0xb44a343 (0xf3a6 bytes) 2024-09-21T22:51:23.471 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:23.464+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:23.471 INFO:teuthology.orchestra.run.smithi094.stdout:4435 read 0xb401e21 thru 0xb4028ac (0xa8c bytes) 2024-09-21T22:51:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:4436 punch from 0xbce25c to 0xbd59c4, (0x7768 bytes) 2024-09-21T22:51:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.472 INFO:teuthology.orchestra.run.smithi094.stdout:4443 write 0xcacabab thru 0xcad8394 (0xd7ea bytes) 2024-09-21T22:51:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:4444 punch from 0x9de3f54 to 0x9df3670, (0xf71c bytes) 2024-09-21T22:51:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:4446 write 0x7666e36 thru 0x7670df6 (0x9fc1 bytes) 2024-09-21T22:51:23.477 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.477 INFO:teuthology.orchestra.run.smithi094.stdout:4448 write 0x90659f9 thru 0x906ab15 (0x511d bytes) 2024-09-21T22:51:23.478 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:23.478 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:23.478 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:23.479 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:23.479 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:23.480 INFO:teuthology.orchestra.run.smithi094.stdout:4450 read 0x316ffe2 thru 0x317672e (0x674d bytes) 2024-09-21T22:51:23.489 INFO:teuthology.orchestra.run.smithi094.stdout:4451 trunc from 0xe71f84c to 0x9acd8cd 2024-09-21T22:51:23.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:23.511 INFO:teuthology.orchestra.run.smithi094.stdout:4453 read 0x7efe774 thru 0x7f032bb (0x4b48 bytes) 2024-09-21T22:51:23.511 INFO:teuthology.orchestra.run.smithi094.stdout:4454 write 0x21f7586 thru 0x21f7f48 (0x9c3 bytes) 2024-09-21T22:51:23.515 INFO:teuthology.orchestra.run.smithi094.stdout:4455 writesame 0x3f950b5 thru 0x3f97bce data_size 0x9(0x2b1a bytes) 2024-09-21T22:51:23.518 INFO:teuthology.orchestra.run.smithi094.stdout:4457 trunc from 0x9acd8cd to 0x97b1a1f 2024-09-21T22:51:23.530 INFO:teuthology.orchestra.run.smithi094.stdout:4459 write 0x2c9b7bd thru 0x2c9f72b (0x3f6f bytes) 2024-09-21T22:51:23.546 INFO:teuthology.orchestra.run.smithi094.stdout:4460 write 0xc5b8d8d thru 0xc5bd213 (0x4487 bytes) 2024-09-21T22:51:23.552 INFO:teuthology.orchestra.run.smithi094.stdout:4461 read 0x9c064b7 thru 0x9c14f2a (0xea74 bytes) 2024-09-21T22:51:23.552 INFO:teuthology.orchestra.run.smithi094.stdout:4462 read 0xeff2ad thru 0xf0d2b3 (0xe007 bytes) 2024-09-21T22:51:23.553 INFO:teuthology.orchestra.run.smithi094.stdout:4463 punch from 0x7b2ab93 to 0x7b35b6b, (0xafd8 bytes) 2024-09-21T22:51:23.554 INFO:teuthology.orchestra.run.smithi094.stdout:4464 write 0x168cb53 thru 0x168d18f (0x63d bytes) 2024-09-21T22:51:23.555 INFO:teuthology.orchestra.run.smithi094.stdout:4466 read 0xcb145 thru 0xd42c6 (0x9182 bytes) 2024-09-21T22:51:23.555 INFO:teuthology.orchestra.run.smithi094.stdout:4468 writesame 0x122e8ec thru 0x123c0cc data_size 0x7(0xd7e1 bytes) 2024-09-21T22:51:23.561 INFO:teuthology.orchestra.run.smithi094.stdout:4470 punch from 0x6c54a84 to 0x6c58825, (0x3da1 bytes) 2024-09-21T22:51:23.561 INFO:teuthology.orchestra.run.smithi094.stdout:4471 writesame 0x97c7894 thru 0x97d2c5b data_size 0x2c(0xb3c8 bytes) 2024-09-21T22:51:23.564 INFO:teuthology.orchestra.run.smithi094.stdout:4472 read 0x5703390 thru 0x5711869 (0xe4da bytes) 2024-09-21T22:51:23.565 INFO:teuthology.orchestra.run.smithi094.stdout:4474 writesame 0x4c0bd47 thru 0x4c184d6 data_size 0x7c(0xc790 bytes) 2024-09-21T22:51:23.569 INFO:teuthology.orchestra.run.smithi094.stdout:4475 read 0x2f5ce14 thru 0x2f61079 (0x4266 bytes) 2024-09-21T22:51:23.572 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:23.564+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:23.582 INFO:teuthology.orchestra.run.smithi094.stdout:4478 trunc from 0xc5bd214 to 0x3e80c42 2024-09-21T22:51:23.609 INFO:teuthology.orchestra.run.smithi094.stdout:4479 write 0x1360ce0 thru 0x136a8bb (0x9bdc bytes) 2024-09-21T22:51:23.636 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:23.636 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.407989572999995, 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:23.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:23.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:23.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.591426575, 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.414461437, 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36671 3.0 e53/24 3:03a43115:::rbd_data.111e4aac08c7.00000000000001e6:head v 53'401, mlcod=53'397)", 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.392643982000003, 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003075979, 2024-09-21T22:51:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242575+0000", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5670000000000002e-06 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242594+0000", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8448999999999999e-05 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242636+0000", 2024-09-21T22:51:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2756999999999997e-05 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242656+0000", 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9643000000000001e-05 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242810+0000", 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000153523 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242868+0000", 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8582000000000003e-05 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245513+0000", 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026443489999999998 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245649+0000", 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013610900000000001 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.803352781000001, 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.623547582, 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.545927911, 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.545662640000003, 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.215589842999997, 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.034503893, 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.019888526999999, 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.019505936000002, 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.017493472000002, 2024-09-21T22:51:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.988768671999999, 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.986075902, 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:107624 3.9 e70/24 3:9604045b:::rbd_data.111eaf8fb188.0000000000000009:head v 70'215, mlcod=68'213)", 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.633642009000001, 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029782390000000001, 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1300000000000001e-07 2024-09-21T22:51:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002740+0000", 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001164691 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002751+0000", 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0978000000000001e-05 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002759+0000", 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1820000000000006e-06 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002838+0000", 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.962e-05 2024-09-21T22:51:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002868+0000", 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9691000000000001e-05 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004502+0000", 2024-09-21T22:51:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001633858 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004553+0000", 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0905999999999997e-05 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147260 3.4 e79/24 3:26d7bf4e:::rbd_data.111e2790987.0000000000000004:head v 79'490, mlcod=74'488)", 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.2262450469999999, 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030032000000000001, 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409517+0000", 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000545047 2024-09-21T22:51:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409532+0000", 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4669e-05 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409540+0000", 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6670000000000002e-06 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409607+0000", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6264000000000001e-05 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409636+0000", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9166000000000002e-05 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411936+0000", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022996959999999999 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411975+0000", 2024-09-21T22:51:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9251000000000001e-05 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.2256162780000004, 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147851 3.4 e79/24 3:26d7bf4e:::rbd_data.111e2790987.0000000000000004:head v 79'493, mlcod=79'491)", 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.5111002340000002, 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031641519999999999, 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124118+0000", 2024-09-21T22:51:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8649999999999999e-06 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124128+0000", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8840000000000006e-06 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124147+0000", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8773000000000001e-05 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124156+0000", 2024-09-21T22:51:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2450000000000004e-06 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124484+0000", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032791899999999998 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124504+0000", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127232+0000", 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027278160000000001 2024-09-21T22:51:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127281+0000", 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8882999999999999e-05 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147845 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'260, mlcod=79'258)", 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.5110017469999999, 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030302570000000002, 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4799999999999998e-07 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124221+0000", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0919999999999997e-06 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124234+0000", 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3261999999999999e-05 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124240+0000", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3690000000000003e-06 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124316+0000", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5789000000000002e-05 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124339+0000", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3042000000000001e-05 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127195+0000", 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028561580000000001 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127245+0000", 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9997000000000003e-05 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:23.728 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:23.729 INFO:teuthology.orchestra.run.smithi094.stdout:4480 write 0xd63fc23 thru 0xd6434ca (0x38a8 bytes) 2024-09-21T22:51:23.729 INFO:teuthology.orchestra.run.smithi094.stdout:4482 read 0xc5fe7f1 thru 0xc60e0a0 (0xf8b0 bytes) 2024-09-21T22:51:23.730 INFO:teuthology.orchestra.run.smithi094.stdout:4484 write 0xc6bb80b thru 0xc6c169d (0x5e93 bytes) 2024-09-21T22:51:23.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4487 trunc from 0xd6434cb to 0xc8c896a 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4489 writesame 0x4ec842d thru 0x4ed19ca data_size 0xb(0x959e bytes) 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4490 write 0xa178f7 thru 0xa1cf28 (0x5632 bytes) 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4492 write 0xc0f2ac9 thru 0xc0f58eb (0x2e23 bytes) 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4493 punch from 0x977c74f to 0x978c3ca, (0xfc7b bytes) 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4494 punch from 0x327ff60 to 0x3287f35, (0x7fd5 bytes) 2024-09-21T22:51:23.732 INFO:teuthology.orchestra.run.smithi094.stdout:4496 trunc from 0xc8c896a to 0x804cd58 2024-09-21T22:51:23.733 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:23.733 INFO:teuthology.orchestra.run.smithi094.stdout:4500 trunc from 0x804cd58 to 0x9372b86 2024-09-21T22:51:23.733 INFO:teuthology.orchestra.run.smithi094.stdout:4501 clone 29 order 19 su 8192 sc 11 2024-09-21T22:51:23.733 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:23.727+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:23.830 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:23.827+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:23.862 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:23.863 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:23.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:23.962 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:23.959+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:24.032 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:24.028+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:24.042 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:24.042 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:24.042 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:24.042 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:24.042 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:24.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:24.146 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:24.143+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:24.227 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21056 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246165116, 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055614907999999998, 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21056 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979243+0000", 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3e-07 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979247+0000", 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6200000000000001e-06 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979268+0000", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1226e-05 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979278+0000", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0212e-05 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034644+0000", 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055365960999999998 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034749+0000", 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104849 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034857+0000", 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010831 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246118158000002, 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246088157000003, 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246060450999998, 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246014500999998, 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245974365999999, 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245944367, 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245910305999999, 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245866450999998, 2024-09-21T22:51:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245830320000003, 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245779739, 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245674524999998, 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245611015000001, 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245512812999998, 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245447550000002, 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245381633000001, 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245304531000002, 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.245230931000002, 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.360130007, 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0812694309999999, 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.313 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:24.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:24.314 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:24.311+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:24.336 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:51:24.415 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:24.408+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:24.495 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:24.495 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:24.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:24.532 INFO:teuthology.orchestra.run.smithi094.stdout:flattening image image_client.0-clone28 2024-09-21T22:51:24.553 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:24.551+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:24.616 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:24.611+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:24.631 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:24.631 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:24.631 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:24.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:24.632 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:24.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:24.716 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:24.712+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:24.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:24.817 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:24.815+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5439637610000005, 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5439219150000003, 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5438764410000001, 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5438415130000003, 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5438092670000003, 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5437735349999997, 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5437363590000004, 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5437024659999992, 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133720 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3160064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3204240630000008, 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058124787999999997, 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133720 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500054+0000", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.094e-06 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500071+0000", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7147999999999999e-05 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500093+0000", 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2184999999999999e-05 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500107+0000", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4151999999999999e-05 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.557904+0000", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057797121 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558040+0000", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135811 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558178+0000", 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013727699999999999 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133725 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3256320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3203505910000004, 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058232719000000002, 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133725 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500127+0000", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1370000000000001e-06 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500133+0000", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4779999999999999e-06 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500153+0000", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0035999999999999e-05 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500166+0000", 2024-09-21T22:51:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3174e-05 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558224+0000", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058057927000000002 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558283+0000", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8928000000000002e-05 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558359+0000", 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6038999999999995e-05 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133732 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3170304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3202929000000001, 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058362280000000002, 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133732 2024-09-21T22:51:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500185+0000", 2024-09-21T22:51:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.145e-06 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500191+0000", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9000000000000003e-06 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500212+0000", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0642e-05 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500225+0000", 2024-09-21T22:51:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3349e-05 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558408+0000", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058183286000000001 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558471+0000", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2446999999999994e-05 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558546+0000", 2024-09-21T22:51:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5511000000000003e-05 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133734 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3192832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3202335820000002, 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058509809000000003, 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133734 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500244+0000", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1689999999999999e-06 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500251+0000", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3820000000000001e-06 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500272+0000", 2024-09-21T22:51:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1671e-05 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500285+0000", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2934e-05 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558588+0000", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058302362000000003 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558678+0000", 2024-09-21T22:51:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9846999999999995e-05 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558753+0000", 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5444000000000005e-05 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133743 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3207168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3201733149999999, 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058675291999999997, 2024-09-21T22:51:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133743 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500305+0000", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.012e-06 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500310+0000", 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1050000000000003e-06 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500331+0000", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1631e-05 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500345+0000", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3988999999999999e-05 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558799+0000", 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058454030999999997 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558882+0000", 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2991000000000006e-05 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558979+0000", 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6533000000000006e-05 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133748 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.320112902, 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058810559999999998, 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133748 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500365+0000", 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500370+0000", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500391+0000", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0707999999999999e-05 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500405+0000", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4209000000000001e-05 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559015+0000", 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05861007 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559108+0000", 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2116999999999999e-05 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559175+0000", 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6999e-05 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133750 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3258368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3200534860000008, 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059112689000000003, 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133750 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500424+0000", 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4900000000000004e-07 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500430+0000", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7019999999999998e-06 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500451+0000", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0763000000000001e-05 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500464+0000", 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3434e-05 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559253+0000", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058788866000000002 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559384+0000", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131335 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559536+0000", 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015164000000000001 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133755 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3172352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3199952899999996, 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059293723999999999, 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133755 2024-09-21T22:51:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500488+0000", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6949999999999997e-06 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500509+0000", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500522+0000", 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3451e-05 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559570+0000", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059047686000000002 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559677+0000", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010658399999999999 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559775+0000", 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8641000000000006e-05 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133765 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3186688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.319936749, 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059623724000000003, 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133765 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500541+0000", 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500547+0000", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5969999999999999e-06 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500567+0000", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9919999999999999e-05 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500580+0000", 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.332e-05 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559852+0000", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059271836000000001 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560015+0000", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016347599999999999 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560164+0000", 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000148515 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133767 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3209216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3198801069999995, 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059822449999999999, 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133767 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500598+0000", 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500603+0000", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6119999999999998e-06 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500623+0000", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0136000000000002e-05 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500637+0000", 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3886000000000001e-05 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560224+0000", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059586443000000003 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560327+0000", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103624 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560419+0000", 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1755000000000005e-05 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133771 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3237888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.3198210110000002, 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060119873999999997, 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133771 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500657+0000", 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7399999999999991e-07 2024-09-21T22:51:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500662+0000", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6540000000000004e-06 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500683+0000", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0414000000000001e-05 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500697+0000", 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3803e-05 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560487+0000", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059790438000000001 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560644+0000", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015662599999999999 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560776+0000", 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131965 2024-09-21T22:51:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5352671220000005, 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:24.915 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:24.916 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T22:51:24.916 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:24.917 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:24.917 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:24.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:24.918 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:24.918 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:24.918 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:24.919 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:24.930 INFO:teuthology.orchestra.run.smithi094.stdout:4502 punch from 0x46f088b to 0x4700640, (0xfdb5 bytes) 2024-09-21T22:51:24.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:24.934 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:24.936 INFO:teuthology.orchestra.run.smithi094.stdout:4505 punch from 0x4343c89 to 0x43538af, (0xfc26 bytes) 2024-09-21T22:51:24.936 INFO:teuthology.orchestra.run.smithi094.stdout:4506 write 0x821f8f3 thru 0x8229d59 (0xa467 bytes) 2024-09-21T22:51:24.951 INFO:teuthology.orchestra.run.smithi094.stdout:4507 read 0x2f9b65a thru 0x2f9c7a8 (0x114f bytes) 2024-09-21T22:51:24.958 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:24.959 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:24.962 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:24.959+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:24.965 INFO:teuthology.orchestra.run.smithi094.stdout:4509 write 0xa263371 thru 0xa272c2e (0xf8be bytes) 2024-09-21T22:51:24.981 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:24.981 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:24.993 INFO:teuthology.orchestra.run.smithi094.stdout:4510 write 0xdd6a28a thru 0xdd760e7 (0xbe5e bytes) 2024-09-21T22:51:25.001 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:25.002 INFO:teuthology.orchestra.run.smithi094.stdout:4511 read 0x824cfba thru 0x825179e (0x47e5 bytes) 2024-09-21T22:51:25.002 INFO:teuthology.orchestra.run.smithi094.stdout:4512 punch from 0x4e9a79b to 0x4ea12b6, (0x6b1b bytes) 2024-09-21T22:51:25.003 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:25.003 INFO:teuthology.orchestra.run.smithi094.stdout:4514 read 0x47312dd thru 0x4737278 (0x5f9c bytes) 2024-09-21T22:51:25.004 INFO:teuthology.orchestra.run.smithi094.stdout:4516 write 0x175a162 thru 0x176886f (0xe70e bytes) 2024-09-21T22:51:25.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:25.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:25.020 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:25.015+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:25.071 INFO:teuthology.orchestra.run.smithi094.stdout:4518 writesame 0xcf13ff3 thru 0xcf22b18 data_size 0x6(0xeb26 bytes) 2024-09-21T22:51:25.084 INFO:teuthology.orchestra.run.smithi094.stdout:4519 write 0x27a882c thru 0x27b2fa1 (0xa776 bytes) 2024-09-21T22:51:25.099 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:25.100 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:25.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:25.149 INFO:teuthology.orchestra.run.smithi094.stdout:4520 trunc from 0xdd760e8 to 0x7b3fcbf 2024-09-21T22:51:25.157 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:25.155+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:25.193 INFO:teuthology.orchestra.run.smithi094.stdout:4521 read 0x51cecc0 thru 0x51dd57c (0xe8bd bytes) 2024-09-21T22:51:25.196 INFO:teuthology.orchestra.run.smithi094.stdout:4522 read 0x2ae9950 thru 0x2af6caf (0xd360 bytes) 2024-09-21T22:51:25.197 INFO:teuthology.orchestra.run.smithi094.stdout:4526 writesame 0x2b9f3ab thru 0x2ba5dfa data_size 0x7e(0x6a50 bytes) 2024-09-21T22:51:25.214 INFO:teuthology.orchestra.run.smithi094.stdout:4527 write 0x909d402 thru 0x909e156 (0xd55 bytes) 2024-09-21T22:51:25.220 INFO:teuthology.orchestra.run.smithi094.stdout:4528 read 0x58cfddb thru 0x58d93fc (0x9622 bytes) 2024-09-21T22:51:25.221 INFO:teuthology.orchestra.run.smithi094.stdout:4529 read 0x3f01836 thru 0x3f09994 (0x815f bytes) 2024-09-21T22:51:25.221 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:25.216+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:25.241 INFO:teuthology.orchestra.run.smithi094.stdout:4530 writesame 0xde9163d thru 0xde93b3b data_size 0x21(0x24ff bytes) 2024-09-21T22:51:25.241 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:25.241 INFO:teuthology.orchestra.run.smithi094.stdout:4533 punch from 0x9585feb to 0x95931e0, (0xd1f5 bytes) 2024-09-21T22:51:25.242 INFO:teuthology.orchestra.run.smithi094.stdout:4534 flatten 2024-09-21T22:51:25.286 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:25.287 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:25.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:25.323 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:25.316+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:25.421 INFO:teuthology.orchestra.run.smithi094.stdout: marking collection for removal 2024-09-21T22:51:25.421 INFO:teuthology.orchestra.run.smithi094.stdout:setting '_remove' omap key 2024-09-21T22:51:25.422 INFO:teuthology.orchestra.run.smithi094.stdout:finish_remove_pgs 3.15_head removing 3.15 2024-09-21T22:51:25.424 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:25.419+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:25.437 INFO:teuthology.orchestra.run.smithi094.stdout:Remove successful 2024-09-21T22:51:25.455 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:25.455 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:25.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:25.525 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:25.520+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:25.625 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:25.623+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:25.651 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:25.651 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:25.666 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:25.757 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:25.755+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:25.828 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:25.823+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:25.849 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:25.849 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:25.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:25.954 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:25.951+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:25.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.15.3 2024-09-21T22:51:26.029 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:26.027+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:26.052 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.052 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:26.130 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:26.127+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:26.231 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:26.227+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:26.238 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.238 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:26.332 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:26.328+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:26.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:51:26.382 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.382 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:26.440 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:26.436+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:26.442 INFO:teuthology.orchestra.run.smithi094.stdout:4535 writesame 0xe54cc48 thru 0xe54e176 data_size 0x11(0x152f bytes) 2024-09-21T22:51:26.442 INFO:teuthology.orchestra.run.smithi094.stdout:4536 punch from 0x4c75278 to 0x4c79b55, (0x48dd bytes) 2024-09-21T22:51:26.442 INFO:teuthology.orchestra.run.smithi094.stdout:4537 read 0xdf4f755 thru 0xdf5d3ec (0xdc98 bytes) 2024-09-21T22:51:26.442 INFO:teuthology.orchestra.run.smithi094.stdout:4539 punch from 0x6cd6e99 to 0x6cd9b27, (0x2c8e bytes) 2024-09-21T22:51:26.443 INFO:teuthology.orchestra.run.smithi094.stdout:4540 read 0x8157101 thru 0x8161c6b (0xab6b bytes) 2024-09-21T22:51:26.443 INFO:teuthology.orchestra.run.smithi094.stdout:4541 read 0xb37f2f3 thru 0xb383c36 (0x4944 bytes) 2024-09-21T22:51:26.443 INFO:teuthology.orchestra.run.smithi094.stdout:4542 write 0x5a7181a thru 0x5a7da64 (0xc24b bytes) 2024-09-21T22:51:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:26.467 INFO:teuthology.orchestra.run.smithi094.stdout:4544 trunc from 0xe54e177 to 0x421f2be 2024-09-21T22:51:26.495 INFO:teuthology.orchestra.run.smithi094.stdout:4545 clone 30 order 22 su 2097152 sc 3 2024-09-21T22:51:26.534 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:26.528+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:26.558 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.558 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:26.656 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:26.652+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:26.731 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.731 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.735 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:26.731+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:26.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:26.836 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:26.832+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:26.894 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:26.894 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:26.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:26.948 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:26.944+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:27.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:27.038 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:27.035+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:27.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:27.139 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:27.135+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:27.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:27.156 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:27.171 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:27.172 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:27.188 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:27.206 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:27.206 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:27.206 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:27.209 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:27.209 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:27.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:27.229 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:27.240 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:27.236+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:27.249 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:27.269 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:27.272 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:27.272 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:27.326 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:27.326 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:27.326 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:27.326 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:27.326 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:27.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:27.340 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:27.336+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:27.345 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:51:27.440 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:27.436+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:27.450 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.221807208000001, 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:27.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:27.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 45.405244209999999, 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:27.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.228279071999999, 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:27.459 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36671 3.0 e53/24 3:03a43115:::rbd_data.111e4aac08c7.00000000000001e6:head v 53'401, mlcod=53'397)", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:27.463 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.206461617000002, 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003075979, 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242573+0000", 2024-09-21T22:51:27.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242575+0000", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5670000000000002e-06 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242594+0000", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8448999999999999e-05 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242636+0000", 2024-09-21T22:51:27.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2756999999999997e-05 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242656+0000", 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9643000000000001e-05 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242810+0000", 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000153523 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.242868+0000", 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8582000000000003e-05 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245513+0000", 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026443489999999998 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.245649+0000", 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013610900000000001 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.617170416, 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:27.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:27.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.437365217, 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:27.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:27.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.359745545999999, 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.359480275000003, 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:27.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.483 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.029407478000003, 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.848321528, 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:27.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:27.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.833706161999999, 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.495 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.833323571000001, 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:27.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.831311107000001, 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.802586306999999, 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:27.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:27.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.799893536999999, 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:27.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:27.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:107624 3.9 e70/24 3:9604045b:::rbd_data.111eaf8fb188.0000000000000009:head v 70'215, mlcod=68'213)", 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.447459643999998, 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029782390000000001, 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.001575+0000", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1300000000000001e-07 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002740+0000", 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001164691 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002751+0000", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0978000000000001e-05 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002759+0000", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1820000000000006e-06 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002838+0000", 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.962e-05 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.002868+0000", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9691000000000001e-05 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004502+0000", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001633858 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:11.004553+0000", 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0905999999999997e-05 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.516 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147260 3.4 e79/24 3:26d7bf4e:::rbd_data.111e2790987.0000000000000004:head v 79'490, mlcod=74'488)", 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.0400626820000003, 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030032000000000001, 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.408972+0000", 2024-09-21T22:51:27.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409517+0000", 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000545047 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409532+0000", 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4669e-05 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409540+0000", 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6670000000000002e-06 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409607+0000", 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6264000000000001e-05 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409636+0000", 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9166000000000002e-05 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411936+0000", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022996959999999999 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411975+0000", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9251000000000001e-05 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:27.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.0394339129999999, 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:27.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:27.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147851 3.4 e79/24 3:26d7bf4e:::rbd_data.111e2790987.0000000000000004:head v 79'493, mlcod=79'491)", 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.3249178690000001, 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031641519999999999, 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124116+0000", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124118+0000", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8649999999999999e-06 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124128+0000", 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8840000000000006e-06 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124147+0000", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8773000000000001e-05 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124156+0000", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2450000000000004e-06 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124484+0000", 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032791899999999998 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124504+0000", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127232+0000", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027278160000000001 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127281+0000", 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8882999999999999e-05 2024-09-21T22:51:27.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147845 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'260, mlcod=79'258)", 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.3248193820000003, 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030302570000000002, 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124215+0000", 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4799999999999998e-07 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124221+0000", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0919999999999997e-06 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124234+0000", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3261999999999999e-05 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124240+0000", 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3690000000000003e-06 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124316+0000", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5789000000000002e-05 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.124339+0000", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3042000000000001e-05 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127195+0000", 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028561580000000001 2024-09-21T22:51:27.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:20.127245+0000", 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9997000000000003e-05 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:27.534 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:27.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:27.564 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T22:51:27.565 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:27.560+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:27.642 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:27.639+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:27.666 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:27.667 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:27.667 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:27.667 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:27.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:27.742 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:27.736+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:27.843 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:27.836+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:27.850 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:27.851 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:27.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:27.916 INFO:teuthology.orchestra.run.smithi094.stdout:4546 write 0xa8418af thru 0xa84e930 (0xd082 bytes) 2024-09-21T22:51:27.922 INFO:teuthology.orchestra.run.smithi094.stdout:4547 punch from 0x5bdd970 to 0x5bdde39, (0x4c9 bytes) 2024-09-21T22:51:27.928 INFO:teuthology.orchestra.run.smithi094.stdout:4548 trunc from 0xa84e931 to 0x54c31a7 2024-09-21T22:51:27.943 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:27.936+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:27.945 INFO:teuthology.orchestra.run.smithi094.stdout:4549 read 0x46d1d91 thru 0x46dd627 (0xb897 bytes) 2024-09-21T22:51:27.950 INFO:teuthology.orchestra.run.smithi094.stdout:4550 trunc from 0x54c31a7 to 0x15167ee 2024-09-21T22:51:27.966 INFO:teuthology.orchestra.run.smithi094.stdout:4551 punch from 0x13366b8 to 0x1336932, (0x27a bytes) 2024-09-21T22:51:27.967 INFO:teuthology.orchestra.run.smithi094.stdout:4552 read 0x40ed2f thru 0x417b7f (0x8e51 bytes) 2024-09-21T22:51:27.978 INFO:teuthology.orchestra.run.smithi094.stdout:4553 write 0x3dc5b3a thru 0x3dcb90a (0x5dd1 bytes) 2024-09-21T22:51:27.994 INFO:teuthology.orchestra.run.smithi094.stdout:4556 write 0x4cb2b0f thru 0x4cb4694 (0x1b86 bytes) 2024-09-21T22:51:28.002 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:28.002 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:28.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21056 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019214949999999, 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055614907999999998, 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21056 2024-09-21T22:51:28.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979242+0000", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979243+0000", 2024-09-21T22:51:28.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3e-07 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979247+0000", 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6200000000000001e-06 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979268+0000", 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1226e-05 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979278+0000", 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0212e-05 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034644+0000", 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055365960999999998 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034749+0000", 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104849 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034857+0000", 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010831 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019167992, 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019137991000001, 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:28.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:28.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:28.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019110284999996, 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019064335000003, 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:28.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:28.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.019024199999997, 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:28.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:28.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018994200999998, 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:28.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018960139999997, 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:28.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018916285000003, 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:28.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018880154000001, 2024-09-21T22:51:28.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:28.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:28.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018829572999998, 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.047 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:28.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018724358999997, 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:28.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018660849, 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018562647000003, 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:28.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:28.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018497384, 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:28.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018431466999999, 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:28.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018354365, 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.018280765, 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1331798409999996, 2024-09-21T22:51:28.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:28.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:28.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.854319265, 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:28.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:28.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.090 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:28.091 INFO:teuthology.orchestra.run.smithi094.stdout:4557 write 0x8e90184 thru 0x8e997ce (0x964b bytes) 2024-09-21T22:51:28.091 INFO:teuthology.orchestra.run.smithi094.stdout:get_pg_num_history pg_num_history pg_num_history(e77 pg_nums {1={17=1},2={20=8},3={24=32}} deleted_pools ) 2024-09-21T22:51:28.091 INFO:teuthology.orchestra.run.smithi094.stdout:Importing pgid 3.15 2024-09-21T22:51:28.092 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8037206:::rbd_data.111e6870f364.0000000000000075:head# 2024-09-21T22:51:28.092 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 1b=[]:{} 2024-09-21T22:51:28.092 INFO:teuthology.orchestra.run.smithi094.stdout:4558 read 0x74be94c thru 0x74c853e (0x9bf3 bytes) 2024-09-21T22:51:28.092 INFO:teuthology.orchestra.run.smithi094.stdout:4560 writesame 0x337597d thru 0x3377af3 data_size 0x1(0x2177 bytes) 2024-09-21T22:51:28.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:28.097 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a818c062:::rbd_data.111e6870f364.0000000000000062:head# 2024-09-21T22:51:28.097 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 1b=[]:{} 2024-09-21T22:51:28.097 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a82b3153:::rbd_data.111eaadcd46c.00000000000002bd:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8314351:::rbd_data.111e7572a6e.0000000000000015:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a842e28b:::rbd_data.111e7572a6e.000000000000002a:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8a5464f:::rbd_data.111e38708da6.0000000000000095:f# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8a5464f:::rbd_data.111e38708da6.0000000000000095:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{f=[f]} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8c34c3c:::rbd_data.111e38708da6.0000000000000069:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a8e5cbc8:::rbd_data.111eaadcd46c.0000000000000015:head# 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a905ddda:::rbd_data.111e4aac08c7.0000000000000331:10# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a905ddda:::rbd_data.111e4aac08c7.0000000000000331:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 10=[]:{10=[10]} 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a92c4996:::rbd_data.111e7572a6e.0000000000000071:6# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a92c4996:::rbd_data.111e7572a6e.0000000000000071:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{6=[6]} 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a94a38c9:::rbd_data.111e33663d62.000000000000006a:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset a=[]:{} 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a952b6c7:::rbd_data.111e9d7ae6fb.000000000000005c:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 15=[]:{} 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a95575fd:::rbd_data.111e7572a6e.000000000000002e:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{} 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a98d08ab:::rbd_data.111e4c6c7d62.0000000000000068:head# 2024-09-21T22:51:28.099 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a9f62cbd:::rbd_data.111e9d7ae6fb.0000000000000091:head# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 15=[]:{} 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:a9fb508b:::rbd_object_map.111eaadcd46c.0000000000000018:head# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:4# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:head# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 4=[]:{4=[4]} 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aabc28af:::rbd_data.111ee1b766.0000000000000005:14# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aabc28af:::rbd_data.111ee1b766.0000000000000005:head# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 14=[]:{14=[14]} 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aadc9c5f:::rbd_data.111e38708da6.0000000000000046:head# 2024-09-21T22:51:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab1b671d:::rbd_data.111e7572a6e.0000000000000061:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab3b4851:::rbd_data.111e33663d62.0000000000000061:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset a=[]:{} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab3d6083:::rbd_data.111e4aac08c7.000000000000002c:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab593a5c:::rbd_data.111e9d7ae6fb.0000000000000077:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 15=[]:{} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab998f4c:::rbd_data.111e38708da6.000000000000021a:f# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ab998f4c:::rbd_data.111e38708da6.000000000000021a:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{f=[f]} 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ac18174b:::rbd_data.111e4c6c7d62.0000000000000008:d# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ac18174b:::rbd_data.111e4c6c7d62.0000000000000008:head# 2024-09-21T22:51:28.101 INFO:teuthology.orchestra.run.smithi094.stdout:snapset d=[]:{d=[d]} 2024-09-21T22:51:28.102 INFO:teuthology.orchestra.run.smithi094.stdout:4562 write 0xd8ec6f1 thru 0xd8fc29c (0xfbac bytes) 2024-09-21T22:51:28.102 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ac275cda:::rbd_header.111e9d7ae6fb:head# 2024-09-21T22:51:28.102 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ac4d23f4:::rbd_data.111e38708da6.00000000000000ab:head# 2024-09-21T22:51:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{} 2024-09-21T22:51:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ac7d6307:::rbd_data.111e752d4c95.0000000000000001:head# 2024-09-21T22:51:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:4563 write 0xb23571e thru 0xb241c39 (0xc51c bytes) 2024-09-21T22:51:28.104 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:acc37f03:::rbd_data.111eaadcd46c.00000000000002df:head# 2024-09-21T22:51:28.104 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.104 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17# 2024-09-21T22:51:28.104 INFO:teuthology.orchestra.run.smithi094.stdout:4564 trunc from 0xd8fc29d to 0xc8cff2d 2024-09-21T22:51:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:head# 2024-09-21T22:51:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 17=[]:{17=[17]} 2024-09-21T22:51:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad189c28:::rbd_data.111e7572a6e.0000000000000046:head# 2024-09-21T22:51:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 6=[]:{} 2024-09-21T22:51:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad246106:::rbd_data.111e38708da6.0000000000000023:f# 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad246106:::rbd_data.111e38708da6.0000000000000023:head# 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{f=[f]} 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad53f7c1:::rbd_data.111eaadcd46c.000000000000017f:head# 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:28.106 INFO:teuthology.orchestra.run.smithi094.stdout:4566 writesame 0xc2e33bd thru 0xc2f263c data_size 0x61(0xf280 bytes) 2024-09-21T22:51:28.107 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad659c07:::rbd_data.111eeb79a41d.000000000000006d:head# 2024-09-21T22:51:28.107 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.107 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad6dc33b:::rbd_data.111e4aac08c7.000000000000007a:10# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad6dc33b:::rbd_data.111e4aac08c7.000000000000007a:head# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 10=[]:{10=[10]} 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad836215:::rbd_data.111e5a344aa.0000000000000032:13# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad836215:::rbd_data.111e5a344aa.0000000000000032:head# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 13=[]:{13=[13]} 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad8798ad:::rbd_data.111e4c6c7d62.000000000000006f:head# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ad97a3a2:::rbd_data.111e38708da6.0000000000000064:head# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:snapset f=[]:{} 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ada6a5f3:::rbd_data.111e9d7ae6fb.0000000000000055:head# 2024-09-21T22:51:28.108 INFO:teuthology.orchestra.run.smithi094.stdout:4567 trunc from 0xc8cff2d to 0xdf28217 2024-09-21T22:51:28.109 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 15=[]:{} 2024-09-21T22:51:28.109 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17# 2024-09-21T22:51:28.109 INFO:teuthology.orchestra.run.smithi094.stdout:4568 trunc from 0xdf28217 to 0xd7f1aff 2024-09-21T22:51:28.109 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:28.084+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:28.110 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:head# 2024-09-21T22:51:28.110 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 17=[]:{17=[17]} 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ae95d2ce:::rbd_data.111ee1b766.0000000000000003:14# 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:ae95d2ce:::rbd_data.111ee1b766.0000000000000003:head# 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 14=[]:{14=[14]} 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aeb2ffa0:::rbd_data.111e4c6c7d62.000000000000008b:head# 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aeb4c717:::rbd_data.111e9d7ae6fb.000000000000003d:head# 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.111 INFO:teuthology.orchestra.run.smithi094.stdout:4570 writesame 0x735e74d thru 0x7369e73 data_size 0x1(0xb727 bytes) 2024-09-21T22:51:28.112 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aec00675:::rbd_data.111ea6ec99f1.0000000000000197:5# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aec00675:::rbd_data.111ea6ec99f1.0000000000000197:head# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 5=[]:{5=[5]} 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aec4f8d7:::rbd_data.111e20a3ef21.0000000000000119:7# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:aec4f8d7:::rbd_data.111e20a3ef21.0000000000000119:head# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 7=[]:{7=[7]} 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af17b25c:::rbd_data.111e33663d62.000000000000006d:head# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:snapset a=[]:{} 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af1c842d:::rbd_data.111ee1b766.000000000000013e:head# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af3e41ae:::rbd_data.111e9d7ae6fb.0000000000000058:head# 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 15=[]:{} 2024-09-21T22:51:28.113 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b# 2024-09-21T22:51:28.114 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af5dc254:::rbd_data.111e6870f364.0000000000000307:head# 2024-09-21T22:51:28.114 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-21T22:51:28.114 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af6d7613:::rbd_data.111edc482e6c.0000000000000000:head# 2024-09-21T22:51:28.114 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.114 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b# 2024-09-21T22:51:28.115 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:head# 2024-09-21T22:51:28.116 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-21T22:51:28.116 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:afce5e93:::rbd_data.111eaadcd46c.00000000000002bb:head# 2024-09-21T22:51:28.116 INFO:teuthology.orchestra.run.smithi094.stdout:snapset 0=[]:{} 2024-09-21T22:51:28.117 INFO:teuthology.orchestra.run.smithi094.stdout:Write #3:afde4ef8:::rbd_data.111e33663d62.0000000000000014:head# 2024-09-21T22:51:28.117 INFO:teuthology.orchestra.run.smithi094.stdout:snapset a=[]:{} 2024-09-21T22:51:28.117 INFO:teuthology.orchestra.run.smithi094.stdout:write_pg epoch 74 info 3.15( v 74'308 (0'0,74'308] local-lis/les=24/25 n=70 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) 2024-09-21T22:51:28.119 INFO:teuthology.orchestra.run.smithi094.stdout:Import successful 2024-09-21T22:51:28.121 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:28.121 INFO:teuthology.orchestra.run.smithi094.stdout:4572 writesame 0x23a424f thru 0x23ac167 data_size 0x1(0x7f19 bytes) 2024-09-21T22:51:28.145 INFO:teuthology.orchestra.run.smithi094.stdout:4573 write 0x7f1e55b thru 0x7f263dd (0x7e83 bytes) 2024-09-21T22:51:28.151 INFO:teuthology.orchestra.run.smithi094.stdout:4574 read 0xd028f3d thru 0xd037481 (0xe545 bytes) 2024-09-21T22:51:28.152 INFO:teuthology.orchestra.run.smithi094.stdout:4575 read 0xbbdbb2 thru 0xbbe2a3 (0x6f2 bytes) 2024-09-21T22:51:28.165 INFO:teuthology.orchestra.run.smithi094.stdout:4576 trunc from 0xd7f1aff to 0x88f167f 2024-09-21T22:51:28.192 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:28.187+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:28.214 INFO:teuthology.orchestra.run.smithi094.stdout:4577 write 0xb33e848 thru 0xb34e121 (0xf8da bytes) 2024-09-21T22:51:28.220 INFO:teuthology.orchestra.run.smithi094.stdout:4578 writesame 0x1798d83 thru 0x17a401f data_size 0x1f(0xb29d bytes) 2024-09-21T22:51:28.224 INFO:teuthology.orchestra.run.smithi094.stdout:4579 punch from 0xb014a4e to 0xb01aa7b, (0x602d bytes) 2024-09-21T22:51:28.227 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:28.227 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:28.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:28.247 INFO:teuthology.orchestra.run.smithi094.stdout:4580 writesame 0xdfabdc8 thru 0xdfb38c4 data_size 0x5(0x7afd bytes) 2024-09-21T22:51:28.247 INFO:teuthology.orchestra.run.smithi094.stdout:4581 read 0x5791d65 thru 0x57947ad (0x2a49 bytes) 2024-09-21T22:51:28.247 INFO:teuthology.orchestra.run.smithi094.stdout:4583 punch from 0x47908bc to 0x479f124, (0xe868 bytes) 2024-09-21T22:51:28.247 INFO:teuthology.orchestra.run.smithi094.stdout:4584 read 0x12a9855 thru 0x12ac1a7 (0x2953 bytes) 2024-09-21T22:51:28.248 INFO:teuthology.orchestra.run.smithi094.stdout:4585 clone 31 order 21 su 16384 sc 3 2024-09-21T22:51:28.329 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:28.327+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:28.394 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:28.391+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:28.411 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:28.411 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:28.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:28.495 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone30 from 0xdfb38c5 (overlap 0x15167ee) to 0x0 2024-09-21T22:51:28.495 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:28.488+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:28.596 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:28.591+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332952345000001, 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332910499, 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332865025, 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332830097, 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332797851, 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332762119, 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332724943000001, 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.332691049999999, 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133720 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3160064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109412646999999, 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058124787999999997, 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133720 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500053+0000", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500054+0000", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.094e-06 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500071+0000", 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7147999999999999e-05 2024-09-21T22:51:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500093+0000", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2184999999999999e-05 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500107+0000", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4151999999999999e-05 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.557904+0000", 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057797121 2024-09-21T22:51:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558040+0000", 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135811 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558178+0000", 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013727699999999999 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133725 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3256320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109339175000001, 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058232719000000002, 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133725 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500126+0000", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500127+0000", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1370000000000001e-06 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500133+0000", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4779999999999999e-06 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500153+0000", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0035999999999999e-05 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500166+0000", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3174e-05 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558224+0000", 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058057927000000002 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558283+0000", 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8928000000000002e-05 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558359+0000", 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6038999999999995e-05 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133732 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3170304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109281484, 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058362280000000002, 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133732 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500184+0000", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500185+0000", 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.145e-06 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500191+0000", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9000000000000003e-06 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500212+0000", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0642e-05 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500225+0000", 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3349e-05 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558408+0000", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058183286000000001 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558471+0000", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2446999999999994e-05 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558546+0000", 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5511000000000003e-05 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133734 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3192832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109222166, 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058509809000000003, 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133734 2024-09-21T22:51:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500243+0000", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500244+0000", 2024-09-21T22:51:28.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1689999999999999e-06 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500251+0000", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3820000000000001e-06 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500272+0000", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1671e-05 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500285+0000", 2024-09-21T22:51:28.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2934e-05 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558588+0000", 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058302362000000003 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558678+0000", 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9846999999999995e-05 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558753+0000", 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5444000000000005e-05 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133743 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3207168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109161899, 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058675291999999997, 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133743 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500304+0000", 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500305+0000", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.012e-06 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500310+0000", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1050000000000003e-06 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500331+0000", 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1631e-05 2024-09-21T22:51:28.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500345+0000", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3988999999999999e-05 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558799+0000", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058454030999999997 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558882+0000", 2024-09-21T22:51:28.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2991000000000006e-05 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.558979+0000", 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6533000000000006e-05 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133748 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109101486, 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058810559999999998, 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133748 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500364+0000", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500365+0000", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.066e-06 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500370+0000", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500391+0000", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0707999999999999e-05 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500405+0000", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4209000000000001e-05 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559015+0000", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05861007 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559108+0000", 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2116999999999999e-05 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559175+0000", 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6999e-05 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133750 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3258368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.109042069999999, 2024-09-21T22:51:28.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059112689000000003, 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133750 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500423+0000", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500424+0000", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4900000000000004e-07 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500430+0000", 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7019999999999998e-06 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500451+0000", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0763000000000001e-05 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500464+0000", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3434e-05 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559253+0000", 2024-09-21T22:51:28.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058788866000000002 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559384+0000", 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131335 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559536+0000", 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015164000000000001 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133755 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3172352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.108983874, 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059293723999999999, 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133755 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500482+0000", 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500488+0000", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6949999999999997e-06 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500509+0000", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500522+0000", 2024-09-21T22:51:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3451e-05 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559570+0000", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059047686000000002 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559677+0000", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010658399999999999 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559775+0000", 2024-09-21T22:51:28.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8641000000000006e-05 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133765 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3186688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.108925333, 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059623724000000003, 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133765 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500540+0000", 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500541+0000", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500547+0000", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5969999999999999e-06 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500567+0000", 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9919999999999999e-05 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500580+0000", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.332e-05 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.559852+0000", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059271836000000001 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560015+0000", 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016347599999999999 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560164+0000", 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000148515 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133767 3.16 3:6fcfd876:::rbd_data.111eaf8fb188.000000000000000a:1a [read 3209216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.108868691, 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059822449999999999, 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133767 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500597+0000", 2024-09-21T22:51:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500598+0000", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500603+0000", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6119999999999998e-06 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500623+0000", 2024-09-21T22:51:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0136000000000002e-05 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500637+0000", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3886000000000001e-05 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560224+0000", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059586443000000003 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560327+0000", 2024-09-21T22:51:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103624 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560419+0000", 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1755000000000005e-05 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:133771 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3237888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.108809595, 2024-09-21T22:51:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060119873999999997, 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133771 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500656+0000", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500657+0000", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7399999999999991e-07 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500662+0000", 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6540000000000004e-06 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500683+0000", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0414000000000001e-05 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.500697+0000", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3803e-05 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560487+0000", 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059790438000000001 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560644+0000", 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015662599999999999 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.560776+0000", 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131965 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.324255706000001, 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:28.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /var/log/ceph/exp.3.15.3 2024-09-21T22:51:28.704 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T22:51:28.704 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:28.705 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:28.699+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:28.804 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:28.799+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:28.875 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:28.875 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:28.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:28.928 INFO:teuthology.orchestra.run.smithi094.stdout:4586 writesame 0xe348c0e thru 0xe34a376 data_size 0x1(0x1769 bytes) 2024-09-21T22:51:28.933 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:28.931+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:28.935 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:28.935 INFO:teuthology.orchestra.run.smithi094.stdout:4588 punch from 0x546e64c to 0x547cdab, (0xe75f bytes) 2024-09-21T22:51:28.935 INFO:teuthology.orchestra.run.smithi094.stdout:4589 punch from 0x9263701 to 0x9271684, (0xdf83 bytes) 2024-09-21T22:51:28.935 INFO:teuthology.orchestra.run.smithi094.stdout:4590 write 0x5a63e4 thru 0x5ad8c0 (0x74dd bytes) 2024-09-21T22:51:29.005 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:29.003+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:29.057 INFO:teuthology.orchestra.run.smithi094.stdout:4591 trunc from 0xe34a377 to 0x762ae22 2024-09-21T22:51:29.066 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:29.066 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:29.066 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:29.066 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:29.066 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:29.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:29.107 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:29.100+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:29.125 INFO:teuthology.orchestra.run.smithi094.stdout:4592 punch from 0x3ba4f3 to 0x3c819a, (0xdca7 bytes) 2024-09-21T22:51:29.141 INFO:teuthology.orchestra.run.smithi094.stdout:4593 writesame 0x99e7dce thru 0x99eaff1 data_size 0x2(0x3224 bytes) 2024-09-21T22:51:29.141 INFO:teuthology.orchestra.run.smithi094.stdout:4595 write 0x26c31cd thru 0x26c4773 (0x15a7 bytes) 2024-09-21T22:51:29.153 INFO:teuthology.orchestra.run.smithi094.stdout:4596 writesame 0x6a923f0 thru 0x6a98436 data_size 0x7(0x6047 bytes) 2024-09-21T22:51:29.214 INFO:teuthology.orchestra.run.smithi094.stdout:4597 trunc from 0x99eaff2 to 0xa7b663a 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4598 trunc from 0xa7b663a to 0x9b75029 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4599 read 0x77482b5 thru 0x774c8cd (0x4619 bytes) 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4600 write 0x8f98f06 thru 0x8f99ff2 (0x10ed bytes) 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4601 punch from 0x2bf39f7 to 0x2bf91d0, (0x57d9 bytes) 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4603 read 0xa80080 thru 0xa8aea9 (0xae2a bytes) 2024-09-21T22:51:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:4605 trunc from 0x9b75029 to 0x2d301e0 2024-09-21T22:51:29.215 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:29.208+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:29.251 INFO:teuthology.orchestra.run.smithi094.stdout:4606 read 0x18f3abe thru 0x18fc733 (0x8c76 bytes) 2024-09-21T22:51:29.258 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.319701577000004, 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774629718, 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774566268000001, 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774503090000003, 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774446750000003, 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774405008000002, 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774344407000001, 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774263971000003, 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774187861000001, 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774100787999998, 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774057300000003, 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:29.331 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.774023661999998, 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:29.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.773989896000003, 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:29.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.773958612000001, 2024-09-21T22:51:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:29.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:29.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:29.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:29.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.773913256999997, 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:29.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.773881760999998, 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:29.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:29.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.773846980000002, 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:29.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:29.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:29.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.755901449, 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.755840405000001, 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:29.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:29.377 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.4332602550000004, 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:29.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:51:29.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:51:29.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:29.385 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:29.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:29.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:51:29.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:29.390 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:29.383+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:4607 write 0x83a64d4 thru 0x83a946f (0x2f9c bytes) 2024-09-21T22:51:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:4609 read 0x49fe7be thru 0x4a0294b (0x418e bytes) 2024-09-21T22:51:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:4610 trunc from 0x83a9470 to 0x8ce3b29 2024-09-21T22:51:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:4611 write 0x88316c1 thru 0x8841535 (0xfe75 bytes) 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4613 write 0xc657450 thru 0xc6615c9 (0xa17a bytes) 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4614 writesame 0xb97e689 thru 0xb981fd0 data_size 0x5e(0x3948 bytes) 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4615 trunc from 0xc6615ca to 0xc9daa30 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4617 punch from 0x328bbfd to 0x329b749, (0xfb4c bytes) 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4618 writesame 0x879a046 thru 0x87a02b0 data_size 0x1(0x626b bytes) 2024-09-21T22:51:29.392 INFO:teuthology.orchestra.run.smithi094.stdout:4620 trunc from 0xc9daa30 to 0x7e3145a 2024-09-21T22:51:29.413 INFO:teuthology.orchestra.run.smithi094.stdout:4621 writesame 0x4e9070a thru 0x4e9cb91 data_size 0x4c(0xc488 bytes) 2024-09-21T22:51:29.446 INFO:teuthology.orchestra.run.smithi094.stdout:4622 write 0xcc08f59 thru 0xcc142b5 (0xb35d bytes) 2024-09-21T22:51:29.458 INFO:teuthology.orchestra.run.smithi094.stdout:4623 punch from 0x1e4d72b to 0x1e5a77c, (0xd051 bytes) 2024-09-21T22:51:29.458 INFO:teuthology.orchestra.run.smithi094.stdout:4625 writesame 0xc35a70b thru 0xc36649f data_size 0x1(0xbd95 bytes) 2024-09-21T22:51:29.481 INFO:teuthology.orchestra.run.smithi094.stdout:4627 writesame 0xbfdf1 thru 0xc76fa data_size 0x1(0x790a bytes) 2024-09-21T22:51:29.488 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:29.484+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:29.494 INFO:teuthology.orchestra.run.smithi094.stdout:4628 read 0x83f56a7 thru 0x8404166 (0xeac0 bytes) 2024-09-21T22:51:29.495 INFO:teuthology.orchestra.run.smithi094.stdout:4630 read 0x1bc70dd thru 0x1bd42db (0xd1ff bytes) 2024-09-21T22:51:29.495 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.509 INFO:teuthology.orchestra.run.smithi094.stdout:4635 write 0xeadc6af thru 0xeadf910 (0x3262 bytes) 2024-09-21T22:51:29.514 INFO:teuthology.orchestra.run.smithi094.stdout:4636 read 0x6dd167f thru 0x6ddb7de (0xa160 bytes) 2024-09-21T22:51:29.538 INFO:teuthology.orchestra.run.smithi094.stdout:4637 trunc from 0xeadf911 to 0xcad3d1 2024-09-21T22:51:29.557 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:29.585 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:29.585 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:29.589 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:29.583+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:29.590 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:29.590 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:29.602 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:29.608 INFO:teuthology.orchestra.run.smithi094.stdout:4639 read 0x48dcb5 thru 0x495bbf (0x7f0b bytes) 2024-09-21T22:51:29.612 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:29.612 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:29.625 INFO:teuthology.orchestra.run.smithi094.stdout:4640 trunc from 0xcad3d1 to 0x9b00f98 2024-09-21T22:51:29.631 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:29.631 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:29.633 INFO:teuthology.orchestra.run.smithi094.stdout:4643 read 0x2cd9dd0 thru 0x2cdef01 (0x5132 bytes) 2024-09-21T22:51:29.649 INFO:teuthology.orchestra.run.smithi094.stdout:4644 write 0xde3d2dc thru 0xde3d45a (0x17f bytes) 2024-09-21T22:51:29.650 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:29.656 INFO:teuthology.orchestra.run.smithi094.stdout:4646 write 0x4e4726b thru 0x4e47717 (0x4ad bytes) 2024-09-21T22:51:29.660 INFO:teuthology.orchestra.run.smithi094.stdout:4648 writesame 0x75e6332 thru 0x75ea559 data_size 0x49(0x4228 bytes) 2024-09-21T22:51:29.667 INFO:teuthology.orchestra.run.smithi094.stdout:4649 write 0x14de4e8 thru 0x14ec55c (0xe075 bytes) 2024-09-21T22:51:29.668 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:29.674 INFO:teuthology.orchestra.run.smithi094.stdout:4651 write 0x1a60b40 thru 0x1a6e225 (0xd6e6 bytes) 2024-09-21T22:51:29.685 INFO:teuthology.orchestra.run.smithi094.stdout:4652 trunc from 0xde3d45b to 0xb194d16 2024-09-21T22:51:29.687 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:29.691 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:29.691 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:29.697 INFO:teuthology.orchestra.run.smithi094.stdout:4653 writesame 0x1bc4341 thru 0x1bc9085 data_size 0x1(0x4d45 bytes) 2024-09-21T22:51:29.697 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:29.695+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:29.707 INFO:teuthology.orchestra.run.smithi094.stdout:4654 read 0xab51515 thru 0xab5f6f8 (0xe1e4 bytes) 2024-09-21T22:51:29.707 INFO:teuthology.orchestra.run.smithi094.stdout:4655 read 0x5de91d8 thru 0x5df83ba (0xf1e3 bytes) 2024-09-21T22:51:29.707 INFO:teuthology.orchestra.run.smithi094.stdout:4656 write 0xa818b1 thru 0xa821a1 (0x8f1 bytes) 2024-09-21T22:51:29.734 INFO:teuthology.orchestra.run.smithi094.stdout:4657 writesame 0x29148f thru 0x29f856 data_size 0x94(0xe3c8 bytes) 2024-09-21T22:51:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:4658 write 0x856f1b4 thru 0x8575f9e (0x6deb bytes) 2024-09-21T22:51:29.745 INFO:teuthology.orchestra.run.smithi094.stdout:4659 read 0xad90d88 thru 0xad97357 (0x65d0 bytes) 2024-09-21T22:51:29.745 INFO:teuthology.orchestra.run.smithi094.stdout:4660 writesame 0xff0da4 thru 0xffe16a data_size 0x7(0xd3c7 bytes) 2024-09-21T22:51:29.750 INFO:teuthology.orchestra.run.smithi094.stdout:4661 read 0x7dd303c thru 0x7de2c86 (0xfc4b bytes) 2024-09-21T22:51:29.750 INFO:teuthology.orchestra.run.smithi094.stdout:4662 write 0x2372e5c thru 0x237dbd5 (0xad7a bytes) 2024-09-21T22:51:29.757 INFO:teuthology.orchestra.run.smithi094.stdout:4664 read 0x66764c thru 0x669b0f (0x24c4 bytes) 2024-09-21T22:51:29.758 INFO:teuthology.orchestra.run.smithi094.stdout:4665 read 0x2766593 thru 0x27760f8 (0xfb66 bytes) 2024-09-21T22:51:29.764 INFO:teuthology.orchestra.run.smithi094.stdout:4668 writesame 0xb3b4abd thru 0xb3bd3ff data_size 0x35(0x8943 bytes) 2024-09-21T22:51:29.768 INFO:teuthology.orchestra.run.smithi094.stdout:4671 trunc from 0xb3bd400 to 0xe16ab6c 2024-09-21T22:51:29.776 INFO:teuthology.orchestra.run.smithi094.stdout:4672 punch from 0x741ead3 to 0x7422a37, (0x3f64 bytes) 2024-09-21T22:51:29.776 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.777 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.777 INFO:teuthology.orchestra.run.smithi094.stdout:4675 write 0x9cb0fda thru 0x9cbdbfd (0xcc24 bytes) 2024-09-21T22:51:29.785 INFO:teuthology.orchestra.run.smithi094.stdout:4676 punch from 0xf3573f to 0xf41c0c, (0xc4cd bytes) 2024-09-21T22:51:29.785 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.785 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:29.785 INFO:teuthology.orchestra.run.smithi094.stdout:4680 writesame 0xcdedb95 thru 0xcdeef30 data_size 0xa(0x139c bytes) 2024-09-21T22:51:29.790 INFO:teuthology.orchestra.run.smithi094.stdout:4681 read 0x667dc90 thru 0x6688386 (0xa6f7 bytes) 2024-09-21T22:51:29.790 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:29.784+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:29.794 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:29.794 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:29.808 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:29.809 INFO:teuthology.orchestra.run.smithi094.stdout:4682 trunc from 0xe16ab6c to 0x14f6758 2024-09-21T22:51:29.836 INFO:teuthology.orchestra.run.smithi094.stdout:4684 punch from 0x27b020 to 0x284353, (0x9333 bytes) 2024-09-21T22:51:29.836 INFO:teuthology.orchestra.run.smithi094.stdout:4685 read 0xe78c3e thru 0xe81703 (0x8ac6 bytes) 2024-09-21T22:51:29.837 INFO:teuthology.orchestra.run.smithi094.stdout:4686 punch from 0x7ae998 to 0x7bc94d, (0xdfb5 bytes) 2024-09-21T22:51:29.837 INFO:teuthology.orchestra.run.smithi094.stdout:4687 write 0x7295aa thru 0x7378f7 (0xe34e bytes) 2024-09-21T22:51:29.885 INFO:teuthology.orchestra.run.smithi094.stdout:4688 read 0xb1a342 thru 0xb241ef (0x9eae bytes) 2024-09-21T22:51:29.886 INFO:teuthology.orchestra.run.smithi094.stdout:4689 punch from 0xe3c8d1 to 0xe43d04, (0x7433 bytes) 2024-09-21T22:51:29.886 INFO:teuthology.orchestra.run.smithi094.stdout:4690 punch from 0xd2014d to 0xd205e6, (0x499 bytes) 2024-09-21T22:51:29.886 INFO:teuthology.orchestra.run.smithi094.stdout:4691 read 0x1271cb0 thru 0x1275671 (0x39c2 bytes) 2024-09-21T22:51:29.892 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:29.888+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:29.924 INFO:teuthology.orchestra.run.smithi094.stdout:4692 write 0xb0140c2 thru 0xb01d982 (0x98c1 bytes) 2024-09-21T22:51:29.952 INFO:teuthology.orchestra.run.smithi094.stdout:4693 write 0xdf4dccc thru 0xdf55969 (0x7c9e bytes) 2024-09-21T22:51:29.977 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:29.977 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:29.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:29.992 INFO:teuthology.orchestra.run.smithi094.stdout:4694 trunc from 0xdf5596a to 0x1c4e69f 2024-09-21T22:51:29.993 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:29.987+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:30.038 INFO:teuthology.orchestra.run.smithi094.stdout:4695 trunc from 0x1c4e69f to 0xdc6c07e 2024-09-21T22:51:30.052 INFO:teuthology.orchestra.run.smithi094.stdout:4696 read 0xabfbc7d thru 0xac071d6 (0xb55a bytes) 2024-09-21T22:51:30.052 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.053 INFO:teuthology.orchestra.run.smithi094.stdout:4699 write 0xd961 thru 0x19ab2 (0xc152 bytes) 2024-09-21T22:51:30.071 INFO:teuthology.orchestra.run.smithi094.stdout:4700 trunc from 0xdc6c07e to 0x4ad3644 2024-09-21T22:51:30.085 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.094 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:30.088+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:30.102 INFO:teuthology.orchestra.run.smithi094.stdout:4702 write 0x6d961dc thru 0x6da60b1 (0xfed6 bytes) 2024-09-21T22:51:30.125 INFO:teuthology.orchestra.run.smithi094.stdout:4703 trunc from 0x6da60b2 to 0x1556234 2024-09-21T22:51:30.130 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.130 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:30.157 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.157 INFO:teuthology.orchestra.run.smithi094.stdout:4705 punch from 0xb95f7d to 0xb9e11d, (0x81a0 bytes) 2024-09-21T22:51:30.183 INFO:teuthology.orchestra.run.smithi094.stdout:4707 trunc from 0x1556234 to 0xcaebc44 2024-09-21T22:51:30.194 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:30.191+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:30.198 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.198 INFO:teuthology.orchestra.run.smithi094.stdout:4711 writesame 0xc1f8e5d thru 0xc1fb61e data_size 0x7(0x27c2 bytes) 2024-09-21T22:51:30.204 INFO:teuthology.orchestra.run.smithi094.stdout:4712 punch from 0xafe345e to 0xaff028a, (0xce2c bytes) 2024-09-21T22:51:30.221 INFO:teuthology.orchestra.run.smithi094.stdout:4713 trunc from 0xcaebc44 to 0x5747658 2024-09-21T22:51:30.265 INFO:teuthology.orchestra.run.smithi094.stdout:4714 writesame 0x672750d thru 0x6727ff5 data_size 0x15(0xae9 bytes) 2024-09-21T22:51:30.265 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.265 INFO:teuthology.orchestra.run.smithi094.stdout:4717 read 0x6292279 thru 0x629e611 (0xc399 bytes) 2024-09-21T22:51:30.265 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.274 INFO:teuthology.orchestra.run.smithi094.stdout:4719 trunc from 0x6727ff6 to 0xacec3d3 2024-09-21T22:51:30.288 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.288 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.289 INFO:teuthology.orchestra.run.smithi094.stdout:4720 write 0x36de307 thru 0x36e44fa (0x61f4 bytes) 2024-09-21T22:51:30.295 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:30.291+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:30.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:30.305 INFO:teuthology.orchestra.run.smithi094.stdout:4722 read 0x7beee07 thru 0x7bf593c (0x6b36 bytes) 2024-09-21T22:51:30.305 INFO:teuthology.orchestra.run.smithi094.stdout:4723 read 0x969081 thru 0x973fae (0xaf2e bytes) 2024-09-21T22:51:30.308 INFO:teuthology.orchestra.run.smithi094.stdout:4724 write 0x6b7b964 thru 0x6b8b68f (0xfd2c bytes) 2024-09-21T22:51:30.319 INFO:teuthology.orchestra.run.smithi094.stdout:4725 write 0x1866ad1 thru 0x18693f5 (0x2925 bytes) 2024-09-21T22:51:30.328 INFO:teuthology.orchestra.run.smithi094.stdout:4726 write 0x9e56fcb thru 0x9e5c3fb (0x5431 bytes) 2024-09-21T22:51:30.343 INFO:teuthology.orchestra.run.smithi094.stdout:4728 trunc from 0xacec3d3 to 0x8ef29a0 2024-09-21T22:51:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:51:30.379 INFO:teuthology.orchestra.run.smithi094.stdout:4729 trunc from 0x8ef29a0 to 0xdf604e9 2024-09-21T22:51:30.395 INFO:teuthology.orchestra.run.smithi094.stdout:4730 write 0x1601308 thru 0x1603621 (0x231a bytes) 2024-09-21T22:51:30.396 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:30.391+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:30.418 INFO:teuthology.orchestra.run.smithi094.stdout:4731 trunc from 0xdf604e9 to 0x25970dd 2024-09-21T22:51:30.431 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.431 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:30.476 INFO:teuthology.orchestra.run.smithi094.stdout:4733 write 0xe64fbfa thru 0xe65d744 (0xdb4b bytes) 2024-09-21T22:51:30.487 INFO:teuthology.orchestra.run.smithi094.stdout:4734 write 0x742114d thru 0x742a6ab (0x955f bytes) 2024-09-21T22:51:30.495 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:30.488+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:30.501 INFO:teuthology.orchestra.run.smithi094.stdout:4735 write 0x14ec498 thru 0x14eea80 (0x25e9 bytes) 2024-09-21T22:51:30.505 INFO:teuthology.orchestra.run.smithi094.stdout:4736 write 0x92e999d thru 0x92f7806 (0xde6a bytes) 2024-09-21T22:51:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:4737 read 0xb9578e1 thru 0xb96165a (0x9d7a bytes) 2024-09-21T22:51:30.519 INFO:teuthology.orchestra.run.smithi094.stdout:4740 writesame 0x90c1b3d thru 0x90c72d6 data_size 0x1(0x579a bytes) 2024-09-21T22:51:30.529 INFO:teuthology.orchestra.run.smithi094.stdout:4741 read 0xd1f3f09 thru 0xd1f6dbb (0x2eb3 bytes) 2024-09-21T22:51:30.530 INFO:teuthology.orchestra.run.smithi094.stdout:4742 write 0x66b8579 thru 0x66c0d96 (0x881e bytes) 2024-09-21T22:51:30.542 INFO:teuthology.orchestra.run.smithi094.stdout:4743 writesame 0x9ebbd8b thru 0x9ec15c1 data_size 0x1(0x5837 bytes) 2024-09-21T22:51:30.573 INFO:teuthology.orchestra.run.smithi094.stdout:4744 writesame 0xa8c5ab2 thru 0xa8c6d44 data_size 0x5(0x1293 bytes) 2024-09-21T22:51:30.585 INFO:teuthology.orchestra.run.smithi094.stdout:4746 trunc from 0xe65d745 to 0xd601eb0 2024-09-21T22:51:30.596 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:30.591+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:30.599 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.600 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.611 INFO:teuthology.orchestra.run.smithi094.stdout:4747 punch from 0x1381adf to 0x13822a7, (0x7c8 bytes) 2024-09-21T22:51:30.611 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.611 INFO:teuthology.orchestra.run.smithi094.stdout:4749 write 0x3ce8462 thru 0x3cf64b7 (0xe056 bytes) 2024-09-21T22:51:30.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:30.625 INFO:teuthology.orchestra.run.smithi094.stdout:4750 writesame 0x2f85 thru 0x12ad3 data_size 0x5(0xfb4f bytes) 2024-09-21T22:51:30.639 INFO:teuthology.orchestra.run.smithi094.stdout:4751 read 0x92de785 thru 0x92eace4 (0xc560 bytes) 2024-09-21T22:51:30.641 INFO:teuthology.orchestra.run.smithi094.stdout:4752 read 0xc5352c1 thru 0xc5367e4 (0x1524 bytes) 2024-09-21T22:51:30.659 INFO:teuthology.orchestra.run.smithi094.stdout:4753 write 0xecaae3b thru 0xecb4315 (0x94db bytes) 2024-09-21T22:51:30.672 INFO:teuthology.orchestra.run.smithi094.stdout:4754 read 0xbe3a62f thru 0xbe41925 (0x72f7 bytes) 2024-09-21T22:51:30.673 INFO:teuthology.orchestra.run.smithi094.stdout:4755 read 0x9ac6ca thru 0x9ae20d (0x1b44 bytes) 2024-09-21T22:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout:4756 read 0xe604c4f thru 0xe608461 (0x3813 bytes) 2024-09-21T22:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout:4758 read 0xa782433 thru 0xa78ce9c (0xaa6a bytes) 2024-09-21T22:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout:4759 read 0xe213dc5 thru 0xe2201b6 (0xc3f2 bytes) 2024-09-21T22:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout:4761 punch from 0x771ce9 to 0x781974, (0xfc8b bytes) 2024-09-21T22:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout:4764 writesame 0xb2713d5 thru 0xb276c8f data_size 0x23(0x58bb bytes) 2024-09-21T22:51:30.683 INFO:teuthology.orchestra.run.smithi094.stdout:4765 read 0x3c50148 thru 0x3c508d3 (0x78c bytes) 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:4768 read 0x710d5b1 thru 0x7117704 (0xa154 bytes) 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:4769 read 0x4e6e883 thru 0x4e76ca8 (0x8426 bytes) 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:4770 read 0x7d1a019 thru 0x7d23543 (0x952b bytes) 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:4772 read 0xf9bc72 thru 0xfa2d81 (0x7110 bytes) 2024-09-21T22:51:30.685 INFO:teuthology.orchestra.run.smithi094.stdout:4774 write 0xba7b745 thru 0xba86277 (0xab33 bytes) 2024-09-21T22:51:30.697 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:30.692+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:30.705 INFO:teuthology.orchestra.run.smithi094.stdout:4775 trunc from 0xecb4316 to 0xa07d4a6 2024-09-21T22:51:30.740 INFO:teuthology.orchestra.run.smithi094.stdout:4776 flatten 2024-09-21T22:51:30.756 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.756 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.775 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:30.779 INFO:teuthology.orchestra.run.smithi094.stdout:4777 writesame 0xd8b9dc9 thru 0xd8c9734 data_size 0x22(0xf96c bytes) 2024-09-21T22:51:30.780 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.780 INFO:teuthology.orchestra.run.smithi094.stdout:4779 write 0x89ebd23 thru 0x89f95b5 (0xd893 bytes) 2024-09-21T22:51:30.809 INFO:teuthology.orchestra.run.smithi094.stdout:4781 trunc from 0xd8c9735 to 0x2303ec5 2024-09-21T22:51:30.810 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:30.804+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:30.840 INFO:teuthology.orchestra.run.smithi094.stdout:4782 read 0xd04faf thru 0xd0dc71 (0x8cc3 bytes) 2024-09-21T22:51:30.883 INFO:teuthology.orchestra.run.smithi094.stdout:4783 writesame 0xe003855 thru 0xe00ee8a data_size 0x53(0xb636 bytes) 2024-09-21T22:51:30.883 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:30.883 INFO:teuthology.orchestra.run.smithi094.stdout:4786 punch from 0x2df5ee0 to 0x2df7497, (0x15b7 bytes) 2024-09-21T22:51:30.899 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:30.895+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:30.905 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:30.906 INFO:teuthology.orchestra.run.smithi094.stdout:4787 trunc from 0xe00ee8b to 0x21642b2 2024-09-21T22:51:30.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:30.924 INFO:teuthology.orchestra.run.smithi094.stdout:4788 trunc from 0x21642b2 to 0xa64517 2024-09-21T22:51:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:4789 trunc from 0xa64517 to 0x85382c5 2024-09-21T22:51:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:4790 read 0x4242acb thru 0x424c6c1 (0x9bf7 bytes) 2024-09-21T22:51:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:4792 read 0x1d5c0b4 thru 0x1d5f155 (0x30a2 bytes) 2024-09-21T22:51:31.000 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:30.995+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:4793 writesame 0xd0f5523 thru 0xd0f59cf data_size 0x3(0x4ad bytes) 2024-09-21T22:51:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:4794 writesame 0xb6c49aa thru 0xb6c6136 data_size 0x1(0x178d bytes) 2024-09-21T22:51:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:4795 write 0x9626372 thru 0x9628b1f (0x27ae bytes) 2024-09-21T22:51:31.030 INFO:teuthology.orchestra.run.smithi094.stdout:4796 read 0x7cad812 thru 0x7cafd41 (0x2530 bytes) 2024-09-21T22:51:31.030 INFO:teuthology.orchestra.run.smithi094.stdout:4797 write 0x358287e thru 0x358eb5e (0xc2e1 bytes) 2024-09-21T22:51:31.052 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.052 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.057 INFO:teuthology.orchestra.run.smithi094.stdout:4798 write 0xd748d78 thru 0xd749258 (0x4e1 bytes) 2024-09-21T22:51:31.065 INFO:teuthology.orchestra.run.smithi094.stdout:4799 write 0xc77afe thru 0xc7b934 (0x3e37 bytes) 2024-09-21T22:51:31.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:31.073 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.073 INFO:teuthology.orchestra.run.smithi094.stdout:4801 write 0xc79d2fd thru 0xc7ab940 (0xe644 bytes) 2024-09-21T22:51:31.085 INFO:teuthology.orchestra.run.smithi094.stdout:4802 write 0x6b1255b thru 0x6b15d78 (0x381e bytes) 2024-09-21T22:51:31.095 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.095 INFO:teuthology.orchestra.run.smithi094.stdout:4804 write 0xbf7847c thru 0xbf793d4 (0xf59 bytes) 2024-09-21T22:51:31.100 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:31.095+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:31.101 INFO:teuthology.orchestra.run.smithi094.stdout:4805 write 0x2dd4fa1 thru 0x2dd8bb7 (0x3c17 bytes) 2024-09-21T22:51:31.110 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.110 INFO:teuthology.orchestra.run.smithi094.stdout:4808 write 0x885f020 thru 0x8864216 (0x51f7 bytes) 2024-09-21T22:51:31.123 INFO:teuthology.orchestra.run.smithi094.stdout:4809 writesame 0x1e04407 thru 0x1e1407b data_size 0x2b(0xfc75 bytes) 2024-09-21T22:51:31.133 INFO:teuthology.orchestra.run.smithi094.stdout:4810 punch from 0x3d5bd51 to 0x3d5d818, (0x1ac7 bytes) 2024-09-21T22:51:31.133 INFO:teuthology.orchestra.run.smithi094.stdout:4812 read 0x3092849 thru 0x30a0b45 (0xe2fd bytes) 2024-09-21T22:51:31.135 INFO:teuthology.orchestra.run.smithi094.stdout:4814 read 0xaa7e13 thru 0xab7de0 (0xffce bytes) 2024-09-21T22:51:31.136 INFO:teuthology.orchestra.run.smithi094.stdout:4815 write 0x86a9f42 thru 0x86b5544 (0xb603 bytes) 2024-09-21T22:51:31.144 INFO:teuthology.orchestra.run.smithi094.stdout:4816 write 0xc56c7d0 thru 0xc575b88 (0x93b9 bytes) 2024-09-21T22:51:31.156 INFO:teuthology.orchestra.run.smithi094.stdout:4817 read 0x3ef0038 thru 0x3ef4ae1 (0x4aaa bytes) 2024-09-21T22:51:31.157 INFO:teuthology.orchestra.run.smithi094.stdout:4819 writesame 0x8d6f925 thru 0x8d7f1f3 data_size 0x1(0xf8cf bytes) 2024-09-21T22:51:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.201 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:31.195+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:31.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:31.210 INFO:teuthology.orchestra.run.smithi094.stdout:4820 trunc from 0xd749259 to 0x7da64e6 2024-09-21T22:51:31.247 INFO:teuthology.orchestra.run.smithi094.stdout:4821 punch from 0x3e63d3a to 0x3e6d361, (0x9627 bytes) 2024-09-21T22:51:31.247 INFO:teuthology.orchestra.run.smithi094.stdout:4822 write 0x68627aa thru 0x6867c8c (0x54e3 bytes) 2024-09-21T22:51:31.273 INFO:teuthology.orchestra.run.smithi094.stdout:4823 trunc from 0x7da64e6 to 0x418732 2024-09-21T22:51:31.301 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:31.295+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:31.314 INFO:teuthology.orchestra.run.smithi094.stdout:4826 read 0x387d02 thru 0x38bd6a (0x4069 bytes) 2024-09-21T22:51:31.314 INFO:teuthology.orchestra.run.smithi094.stdout:4828 punch from 0x1c389f to 0x1cdfb7, (0xa718 bytes) 2024-09-21T22:51:31.314 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.314 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.331 INFO:teuthology.orchestra.run.smithi094.stdout:4831 trunc from 0x418732 to 0x7c0978e 2024-09-21T22:51:31.345 INFO:teuthology.orchestra.run.smithi094.stdout:4834 read 0x407a05a thru 0x40897ae (0xf755 bytes) 2024-09-21T22:51:31.362 INFO:teuthology.orchestra.run.smithi094.stdout:4835 write 0x886e552 thru 0x8872c23 (0x46d2 bytes) 2024-09-21T22:51:31.371 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.374 INFO:teuthology.orchestra.run.smithi094.stdout:4837 trunc from 0x8872c24 to 0x9c8d0ff 2024-09-21T22:51:31.379 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.379 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:31.399 INFO:teuthology.orchestra.run.smithi094.stdout:4838 trunc from 0x9c8d0ff to 0x5261270 2024-09-21T22:51:31.402 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:31.396+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:31.423 INFO:teuthology.orchestra.run.smithi094.stdout:4839 trunc from 0x5261270 to 0x71676a8 2024-09-21T22:51:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:4841 read 0x4a66913 thru 0x4a688ba (0x1fa8 bytes) 2024-09-21T22:51:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:4842 read 0x7099eaa thru 0x70a730d (0xd464 bytes) 2024-09-21T22:51:31.449 INFO:teuthology.orchestra.run.smithi094.stdout:4843 write 0xa354962 thru 0xa35d1b2 (0x8851 bytes) 2024-09-21T22:51:31.457 INFO:teuthology.orchestra.run.smithi094.stdout:4844 trunc from 0xa35d1b3 to 0x9f8e963 2024-09-21T22:51:31.488 INFO:teuthology.orchestra.run.smithi094.stdout:4845 writesame 0xab004f6 thru 0xab071f9 data_size 0x2(0x6d04 bytes) 2024-09-21T22:51:31.489 INFO:teuthology.orchestra.run.smithi094.stdout:4847 write 0x4acac5a thru 0x4accbcb (0x1f72 bytes) 2024-09-21T22:51:31.496 INFO:teuthology.orchestra.run.smithi094.stdout:4849 read 0x680f5d thru 0x69035e (0xf402 bytes) 2024-09-21T22:51:31.496 INFO:teuthology.orchestra.run.smithi094.stdout:4850 read 0x4b3e70f thru 0x4b4d849 (0xf13b bytes) 2024-09-21T22:51:31.497 INFO:teuthology.orchestra.run.smithi094.stdout:4852 punch from 0x7e32575 to 0x7e3f0da, (0xcb65 bytes) 2024-09-21T22:51:31.497 INFO:teuthology.orchestra.run.smithi094.stdout:4853 write 0x890625c thru 0x890bcaa (0x5a4f bytes) 2024-09-21T22:51:31.502 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:31.496+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:31.506 INFO:teuthology.orchestra.run.smithi094.stdout:4855 write 0x4b8a6fb thru 0x4b9a3a0 (0xfca6 bytes) 2024-09-21T22:51:31.512 INFO:teuthology.orchestra.run.smithi094.stdout:4856 write 0x8047574 thru 0x8048371 (0xdfe bytes) 2024-09-21T22:51:31.534 INFO:teuthology.orchestra.run.smithi094.stdout:4857 write 0xb7e8830 thru 0xb7eb4e1 (0x2cb2 bytes) 2024-09-21T22:51:31.542 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.542 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.556 INFO:teuthology.orchestra.run.smithi094.stdout:4858 writesame 0xc885bf3 thru 0xc886964 data_size 0x1(0xd72 bytes) 2024-09-21T22:51:31.556 INFO:teuthology.orchestra.run.smithi094.stdout:4859 read 0x22d0bae thru 0x22d2ca2 (0x20f5 bytes) 2024-09-21T22:51:31.556 INFO:teuthology.orchestra.run.smithi094.stdout:4860 punch from 0xc547e78 to 0xc557525, (0xf6ad bytes) 2024-09-21T22:51:31.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:31.572 INFO:teuthology.orchestra.run.smithi094.stdout:4861 trunc from 0xc886965 to 0x6986993 2024-09-21T22:51:31.591 INFO:teuthology.orchestra.run.smithi094.stdout:4862 punch from 0x426311a to 0x426f4a0, (0xc386 bytes) 2024-09-21T22:51:31.606 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:31.603+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:31.614 INFO:teuthology.orchestra.run.smithi094.stdout:4863 write 0xb78e1ac thru 0xb797fed (0x9e42 bytes) 2024-09-21T22:51:31.629 INFO:teuthology.orchestra.run.smithi094.stdout:4864 trunc from 0xb797fee to 0x8c9d655 2024-09-21T22:51:31.649 INFO:teuthology.orchestra.run.smithi094.stdout:4868 read 0xd7684e thru 0xd8592a (0xf0dd bytes) 2024-09-21T22:51:31.649 INFO:teuthology.orchestra.run.smithi094.stdout:4869 read 0x1ad5456 thru 0x1ad876d (0x3318 bytes) 2024-09-21T22:51:31.667 INFO:teuthology.orchestra.run.smithi094.stdout:4870 write 0xa6cb5eb thru 0xa6da66c (0xf082 bytes) 2024-09-21T22:51:31.681 INFO:teuthology.orchestra.run.smithi094.stdout:4871 read 0xdf2eea thru 0xdfa516 (0x762d bytes) 2024-09-21T22:51:31.681 INFO:teuthology.orchestra.run.smithi094.stdout:4872 punch from 0x683f1e5 to 0x684bb19, (0xc934 bytes) 2024-09-21T22:51:31.690 INFO:teuthology.orchestra.run.smithi094.stdout:4873 trunc from 0xa6da66d to 0xd801863 2024-09-21T22:51:31.691 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.691 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:31.704 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:31.699+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:31.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:31.707 INFO:teuthology.orchestra.run.smithi094.stdout:4874 punch from 0x42a6001 to 0x42aa275, (0x4274 bytes) 2024-09-21T22:51:31.707 INFO:teuthology.orchestra.run.smithi094.stdout:4875 read 0xcde60ae thru 0xcde95d0 (0x3523 bytes) 2024-09-21T22:51:31.707 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.727 INFO:teuthology.orchestra.run.smithi094.stdout:4877 trunc from 0xd801863 to 0x54c96bc 2024-09-21T22:51:31.746 INFO:teuthology.orchestra.run.smithi094.stdout:4879 write 0x15582a thru 0x15b4c8 (0x5c9f bytes) 2024-09-21T22:51:31.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.760 INFO:teuthology.orchestra.run.smithi094.stdout:4884 trunc from 0x54c96bc to 0xa23d7d1 2024-09-21T22:51:31.768 INFO:teuthology.orchestra.run.smithi094.stdout:4885 read 0x378c646 thru 0x379025e (0x3c19 bytes) 2024-09-21T22:51:31.769 INFO:teuthology.orchestra.run.smithi094.stdout:4886 writesame 0x4c2823 thru 0x4cd7d3 data_size 0x1(0xafb1 bytes) 2024-09-21T22:51:31.804 INFO:teuthology.orchestra.run.smithi094.stdout:4889 writesame 0x106e61e thru 0x106f1b3 data_size 0x1(0xb96 bytes) 2024-09-21T22:51:31.804 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:31.799+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.810 INFO:teuthology.orchestra.run.smithi094.stdout:4892 trunc from 0xa23d7d1 to 0x9d67b44 2024-09-21T22:51:31.834 INFO:teuthology.orchestra.run.smithi094.stdout:4893 write 0xc14d9c7 thru 0xc157236 (0x9870 bytes) 2024-09-21T22:51:31.841 INFO:teuthology.orchestra.run.smithi094.stdout:4894 trunc from 0xc157237 to 0xbb6c094 2024-09-21T22:51:31.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:31.854 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:31.856 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.858 INFO:teuthology.orchestra.run.smithi094.stdout:4896 trunc from 0xbb6c094 to 0xd849a24 2024-09-21T22:51:31.870 INFO:teuthology.orchestra.run.smithi094.stdout:4897 write 0x1ac749f thru 0x1acde78 (0x69da bytes) 2024-09-21T22:51:31.871 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:31.871 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:31.886 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:31.886 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:31.888 INFO:teuthology.orchestra.run.smithi094.stdout:4899 trunc from 0xd849a24 to 0x7c3d8a8 2024-09-21T22:51:31.889 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:31.890 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:31.890 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:31.900 INFO:teuthology.orchestra.run.smithi094.stdout:4901 writesame 0x24b3966 thru 0x24b8f7b data_size 0x3(0x5616 bytes) 2024-09-21T22:51:31.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:31.905 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:31.900+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:31.911 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:31.911 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:31.911 INFO:teuthology.orchestra.run.smithi094.stdout:4903 write 0x5982a0d thru 0x5992275 (0xf869 bytes) 2024-09-21T22:51:31.920 INFO:teuthology.orchestra.run.smithi094.stdout:4904 read 0x36495b thru 0x36639d (0x1a43 bytes) 2024-09-21T22:51:31.920 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.931 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:31.946 INFO:teuthology.orchestra.run.smithi094.stdout:4906 write 0xdc48166 thru 0xdc50262 (0x80fd bytes) 2024-09-21T22:51:31.950 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:31.957 INFO:teuthology.orchestra.run.smithi094.stdout:4907 write 0x3a49b3b thru 0x3a51830 (0x7cf6 bytes) 2024-09-21T22:51:31.965 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:31.965 INFO:teuthology.orchestra.run.smithi094.stdout:4912 write 0xc8b7e98 thru 0xc8c5394 (0xd4fd bytes) 2024-09-21T22:51:31.970 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:31.970 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:31.973 INFO:teuthology.orchestra.run.smithi094.stdout:4913 writesame 0x4c0b3af thru 0x4c0de5a data_size 0x2(0x2aac bytes) 2024-09-21T22:51:31.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:31.973 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:31.983 INFO:teuthology.orchestra.run.smithi094.stdout:4914 trunc from 0xdc50263 to 0xb683dee 2024-09-21T22:51:31.997 INFO:teuthology.orchestra.run.smithi094.stdout:4915 write 0x812c76e thru 0x8137495 (0xad28 bytes) 2024-09-21T22:51:32.005 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:32.003+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:32.007 INFO:teuthology.orchestra.run.smithi094.stdout:4916 read 0x15edeb thru 0x1614d1 (0x26e7 bytes) 2024-09-21T22:51:32.009 INFO:teuthology.orchestra.run.smithi094.stdout:4918 write 0x119a778 thru 0x119c310 (0x1b99 bytes) 2024-09-21T22:51:32.014 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.020 INFO:teuthology.orchestra.run.smithi094.stdout:4921 trunc from 0xb683dee to 0x8ecbe7d 2024-09-21T22:51:32.028 INFO:teuthology.orchestra.run.smithi094.stdout:4923 writesame 0x223db5d thru 0x2241702 data_size 0xf(0x3ba6 bytes) 2024-09-21T22:51:32.035 INFO:teuthology.orchestra.run.smithi094.stdout:4924 write 0x4c2061 thru 0x4c2b59 (0xaf9 bytes) 2024-09-21T22:51:32.039 INFO:teuthology.orchestra.run.smithi094.stdout:4925 punch from 0x60e233b to 0x60e314a, (0xe0f bytes) 2024-09-21T22:51:32.039 INFO:teuthology.orchestra.run.smithi094.stdout:4928 writesame 0x33759af thru 0x337951d data_size 0x11(0x3b6f bytes) 2024-09-21T22:51:32.045 INFO:teuthology.orchestra.run.smithi094.stdout:4929 read 0x73b1f9a thru 0x73bdc1f (0xbc86 bytes) 2024-09-21T22:51:32.045 INFO:teuthology.orchestra.run.smithi094.stdout:4930 write 0x253869b thru 0x253af3d (0x28a3 bytes) 2024-09-21T22:51:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:4931 read 0x6e8376d thru 0x6e90f4b (0xd7df bytes) 2024-09-21T22:51:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:4932 punch from 0x30fed2e to 0x3108cf2, (0x9fc4 bytes) 2024-09-21T22:51:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.062 INFO:teuthology.orchestra.run.smithi094.stdout:4934 writesame 0xa6860e3 thru 0xa68a50f data_size 0x1(0x442d bytes) 2024-09-21T22:51:32.063 INFO:teuthology.orchestra.run.smithi094.stdout:4935 read 0x59f8761 thru 0x59fcdaf (0x464f bytes) 2024-09-21T22:51:32.064 INFO:teuthology.orchestra.run.smithi094.stdout:4936 read 0xa0456c9 thru 0xa049ca6 (0x45de bytes) 2024-09-21T22:51:32.064 INFO:teuthology.orchestra.run.smithi094.stdout:4938 read 0x825fbc2 thru 0x8263e94 (0x42d3 bytes) 2024-09-21T22:51:32.064 INFO:teuthology.orchestra.run.smithi094.stdout:4940 read 0x6b95946 thru 0x6b9f753 (0x9e0e bytes) 2024-09-21T22:51:32.064 INFO:teuthology.orchestra.run.smithi094.stdout:4941 write 0x47bdeba thru 0x47c718e (0x92d5 bytes) 2024-09-21T22:51:32.076 INFO:teuthology.orchestra.run.smithi094.stdout:4942 writesame 0x16c9ed6 thru 0x16d1011 data_size 0x2(0x713c bytes) 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167180 3.1c 3:3985e20f:::rbd_object_map.111ede3bcd88:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:32.076849+0000", 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0033844880000000002, 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034310650000000001, 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167180 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076849+0000", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076849+0000", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076850+0000", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076857+0000", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2339999999999997e-06 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076876+0000", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8525999999999998e-05 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076885+0000", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1339999999999993e-06 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076963+0000", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8263999999999995e-05 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.077077+0000", 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011362 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.077494+0000", 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000416931 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167181 3.1c 3:3985e20f:::rbd_object_map.111ede3bcd88:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:32.076905+0000", 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003328545, 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034743780000000002, 2024-09-21T22:51:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167181 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076905+0000", 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076905+0000", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076905+0000", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9800000000000003e-07 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076909+0000", 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5779999999999999e-06 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076918+0000", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0059999999999998e-06 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076924+0000", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2859999999999996e-06 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.078266+0000", 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001342427 2024-09-21T22:51:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.078386+0000", 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119252 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.078875+0000", 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048902799999999997 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167182 3.1c 3:3985e20f:::rbd_object_map.111ede3bcd88:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:32.076933+0000", 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0033005529999999999, 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035341449999999998, 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167182 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076933+0000", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076933+0000", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076933+0000", 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4799999999999999e-07 2024-09-21T22:51:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076936+0000", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9440000000000001e-06 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076942+0000", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0839999999999997e-06 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.076947+0000", 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6260000000000003e-06 2024-09-21T22:51:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.079605+0000", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026583430000000001 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.079713+0000", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010835799999999999 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080415+0000", 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000701777 2024-09-21T22:51:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4382.0:167180 3.1c e85/24)", 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:32.080027+0000", 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00020581700000000001, 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054363999999999997, 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080027+0000", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080027+0000", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080028+0000", 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7199999999999997e-07 2024-09-21T22:51:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080033+0000", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8899999999999998e-06 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080044+0000", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0910999999999999e-05 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:32.080051+0000", 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460000000000002e-06 2024-09-21T22:51:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 4 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi094.stdout:4944 punch from 0x6eb2a77 to 0x6eb9e24, (0x73ad bytes) 2024-09-21T22:51:32.097 INFO:teuthology.orchestra.run.smithi094.stdout:4946 write 0x959c1b thru 0x961b6d (0x7f53 bytes) 2024-09-21T22:51:32.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:32.125 INFO:teuthology.orchestra.run.smithi094.stdout:4947 trunc from 0xa68a510 to 0x324a27 2024-09-21T22:51:32.141 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:32.139+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:32.158 INFO:teuthology.orchestra.run.smithi094.stdout:4948 read 0x12782f thru 0x12ccbe (0x5490 bytes) 2024-09-21T22:51:32.207 INFO:teuthology.orchestra.run.smithi094.stdout:4949 writesame 0xcc8b88a thru 0xcc9596b data_size 0x1(0xa0e2 bytes) 2024-09-21T22:51:32.207 INFO:teuthology.orchestra.run.smithi094.stdout:4955 punch from 0x3ecf200 to 0x3ed2438, (0x3238 bytes) 2024-09-21T22:51:32.207 INFO:teuthology.orchestra.run.smithi094.stdout:4956 read 0x269372 thru 0x26a42d (0x10bc bytes) 2024-09-21T22:51:32.207 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:32.200+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:32.208 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.208 INFO:teuthology.orchestra.run.smithi094.stdout:4959 punch from 0x93edb48 to 0x93fac2f, (0xd0e7 bytes) 2024-09-21T22:51:32.208 INFO:teuthology.orchestra.run.smithi094.stdout:4960 writesame 0x4c7d93d thru 0x4c7e9f6 data_size 0x1(0x10ba bytes) 2024-09-21T22:51:32.217 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.217 INFO:teuthology.orchestra.run.smithi094.stdout:4962 writesame 0x4931f47 thru 0x49337a3 data_size 0x3f(0x185d bytes) 2024-09-21T22:51:32.222 INFO:teuthology.orchestra.run.smithi094.stdout:4964 read 0xae6465 thru 0xaec5be (0x615a bytes) 2024-09-21T22:51:32.237 INFO:teuthology.orchestra.run.smithi094.stdout:4965 write 0xd48ba5a thru 0xd48cb3a (0x10e1 bytes) 2024-09-21T22:51:32.246 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.246 INFO:teuthology.orchestra.run.smithi094.stdout:4967 read 0xa890e3c thru 0xa89ab40 (0x9d05 bytes) 2024-09-21T22:51:32.246 INFO:teuthology.orchestra.run.smithi094.stdout:4968 read 0x83be418 thru 0x83c1497 (0x3080 bytes) 2024-09-21T22:51:32.247 INFO:teuthology.orchestra.run.smithi094.stdout:4969 writesame 0x7c32b53 thru 0x7c37b5d data_size 0x1(0x500b bytes) 2024-09-21T22:51:32.264 INFO:teuthology.orchestra.run.smithi094.stdout:4970 punch from 0xbee90cd to 0xbef2101, (0x9034 bytes) 2024-09-21T22:51:32.264 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:32.271 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:32.272 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:32.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:4972 trunc from 0xd48cb3b to 0x3129618 2024-09-21T22:51:32.330 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:32.327+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:4974 write 0xcc3f49c thru 0xcc4144d (0x1fb2 bytes) 2024-09-21T22:51:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:4975 read 0xc41222a thru 0xc41ffe3 (0xddba bytes) 2024-09-21T22:51:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:4977 trunc from 0xcc4144e to 0xec7afe2 2024-09-21T22:51:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:4979 read 0x6a98f5c thru 0x6a9fbde (0x6c83 bytes) 2024-09-21T22:51:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:4980 write 0xdef4361 thru 0xdefa65b (0x62fb bytes) 2024-09-21T22:51:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:4982 write 0xb607110 thru 0xb613ead (0xcd9e bytes) 2024-09-21T22:51:32.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:51:32.399 INFO:teuthology.orchestra.run.smithi094.stdout:4983 trunc from 0xec7afe2 to 0xa03c075 2024-09-21T22:51:32.408 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:32.404+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:32.417 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137284841, 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137242995000001, 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137197521000001, 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137162592999999, 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137130346999999, 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137094614999999, 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137057438999999, 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.137023546000002, 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.128588202, 2024-09-21T22:51:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164845 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1572864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954175+0000", 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.459623321, 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061736408, 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164845 2024-09-21T22:51:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954175+0000", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954175+0000", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954176+0000", 2024-09-21T22:51:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954180+0000", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.78e-06 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954197+0000", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6646999999999998e-05 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954210+0000", 2024-09-21T22:51:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3443e-05 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.015442+0000", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061231940999999998 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.015534+0000", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1289e-05 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.015911+0000", 2024-09-21T22:51:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037729199999999999 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4595714690000001, 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:51:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:51:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.45951985, 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:51:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:51:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4594685470000002, 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:51:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:51:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4594106309999999, 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:51:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4593549719999999, 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4592988230000001, 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4592363310000001, 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4591355130000001, 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:51:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4590796940000001, 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:51:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4590269880000002, 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:51:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:51:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:51:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:32.510 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:4984 writesame 0xc4d3c61 thru 0xc4d8fd3 data_size 0x1(0x5373 bytes) 2024-09-21T22:51:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:4985 read 0xa9c0cfa thru 0xa9c32ad (0x25b4 bytes) 2024-09-21T22:51:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:4986 clone 32 order 25 su 8388608 sc 9 2024-09-21T22:51:32.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:32.512 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:32.507+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:32.612 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:32.607+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:32.681 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:32.681 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:32.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:32.713 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:32.708+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:32.814 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:32.812+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:32.861 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:32.862 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:32.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:32.922 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:32.919+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 100.063349073, 2024-09-21T22:51:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.518277214000001, 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.518213764000002, 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.518150585999997, 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.518094245999997, 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.518052504000003, 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517991903000002, 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517911466999998, 2024-09-21T22:51:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517835357000003, 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517748284, 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517704795999997, 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517671157999999, 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517637391999997, 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517606108000003, 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517560752999998, 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517529257, 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.517494476000003, 2024-09-21T22:51:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38099 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.499548945000001, 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096642142, 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38099 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499096+0000", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499098+0000", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499103+0000", 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-21T22:51:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499126+0000", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2555999999999999e-05 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499139+0000", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3232e-05 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595494+0000", 2024-09-21T22:51:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096354698000000003 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595623+0000", 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012936999999999999 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595739+0000", 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115449 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.499487901000002, 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.176907751, 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:51:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:51:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.092 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:33.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:33.094 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:33.092+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:33.194 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:33.191+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:33.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:33.265 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:33.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:33.295 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:33.292+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:33.389 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:51:33.395 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:33.391+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:33.452 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:33.453 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:33.468 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:33.510 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:33.507+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:33.597 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:33.591+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.607399862999998, 2024-09-21T22:51:33.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.603162294000001, 2024-09-21T22:51:33.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:33.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:33.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.602967395999997, 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:33.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:33.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:33.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:33.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881292767, 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:51:33.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881250887, 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881215300999999, 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:51:33.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:51:33.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881183118999999, 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:51:33.661 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:51:33.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:51:33.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881151093, 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:51:33.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:51:33.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881081081, 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.881017202000001, 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:51:33.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880963959000001, 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:33.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:51:33.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:51:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880912649999999, 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:51:33.682 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:51:33.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:51:33.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880861825, 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:51:33.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880814665999999, 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:51:33.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:51:33.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880779130000001, 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:51:33.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:51:33.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880730901, 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.700 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:51:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:51:33.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880662759, 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880601623, 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:51:33.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:51:33.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880543455, 2024-09-21T22:51:33.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:33.716 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.880487305999999, 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:33.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:51:33.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:33.721 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T22:51:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:4987 read 0x59a3ade thru 0x59b004e (0xc571 bytes) 2024-09-21T22:51:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:4992 write 0x6eda800 thru 0x6ee7eaf (0xd6b0 bytes) 2024-09-21T22:51:33.722 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:33.716+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:33.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:33.734 INFO:teuthology.orchestra.run.smithi094.stdout:4993 read 0x85afbc2 thru 0x85bf39d (0xf7dc bytes) 2024-09-21T22:51:33.734 INFO:teuthology.orchestra.run.smithi094.stdout:4995 read 0xb477acc thru 0xb48463f (0xcb74 bytes) 2024-09-21T22:51:33.735 INFO:teuthology.orchestra.run.smithi094.stdout:4996 read 0x69712a4 thru 0x697a46f (0x91cc bytes) 2024-09-21T22:51:33.735 INFO:teuthology.orchestra.run.smithi094.stdout:4997 write 0xbb1c002 thru 0xbb29cef (0xdcee bytes) 2024-09-21T22:51:33.764 INFO:teuthology.orchestra.run.smithi094.stdout:4998 trunc from 0xc4d8fd4 to 0x11b543d 2024-09-21T22:51:33.765 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 7, 6] 2024-09-21T22:51:33.765 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-21T22:51:33.765 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T22:51:33.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T22:51:33.767 INFO:tasks.ceph.osd.3:Started 2024-09-21T22:51:33.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:5001 read 0x58e4e1 thru 0x599c86 (0xb7a6 bytes) 2024-09-21T22:51:33.823 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:33.816+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:33.829 INFO:teuthology.orchestra.run.smithi094.stdout:5002 trunc from 0x11b543d to 0xe710bb3 2024-09-21T22:51:33.840 INFO:teuthology.orchestra.run.smithi094.stdout:5003 read 0x558e52f thru 0x5591f07 (0x39d9 bytes) 2024-09-21T22:51:33.841 INFO:teuthology.orchestra.run.smithi094.stdout:5004 punch from 0x2daee92 to 0x2db628c, (0x73fa bytes) 2024-09-21T22:51:33.841 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:33.841 INFO:teuthology.orchestra.run.smithi094.stdout:5006 write 0xa14ee38 thru 0xa15913d (0xa306 bytes) 2024-09-21T22:51:33.844 INFO:teuthology.orchestra.run.smithi094.stdout:5007 write 0x4d5e110 thru 0x4d6afea (0xcedb bytes) 2024-09-21T22:51:33.846 INFO:teuthology.orchestra.run.smithi094.stdout:5008 write 0x4ca82bf thru 0x4cb1fc3 (0x9d05 bytes) 2024-09-21T22:51:33.848 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:33.848 INFO:teuthology.orchestra.run.smithi094.stdout:5010 read 0x120b8d0 thru 0x1215d60 (0xa491 bytes) 2024-09-21T22:51:33.849 INFO:teuthology.orchestra.run.smithi094.stdout:5011 write 0xed9b16 thru 0xee8042 (0xe52d bytes) 2024-09-21T22:51:33.867 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:33.867 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:33.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:33.867 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:33.875 INFO:teuthology.orchestra.run.smithi094.stdout:5012 trunc from 0xe710bb3 to 0xe91a7f 2024-09-21T22:51:33.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:33.896 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:33.909 INFO:teuthology.orchestra.run.smithi094.stdout:5014 trunc from 0xe91a7f to 0x91fa1a6 2024-09-21T22:51:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:config diff 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:config diff get 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:config get 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:config help [] 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:config set ... 2024-09-21T22:51:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:admin_socket: invalid command 2024-09-21T22:51:33.920 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T22:51:33.921 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-21T22:51:33.936 INFO:teuthology.orchestra.run.smithi094.stdout:5015 write 0xc4a5713 thru 0xc4b2109 (0xc9f7 bytes) 2024-09-21T22:51:33.936 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:33.932+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:33.942 INFO:teuthology.orchestra.run.smithi094.stdout:5016 writesame 0x50b09cc thru 0x50b852d data_size 0x11(0x7b62 bytes) 2024-09-21T22:51:33.947 INFO:teuthology.orchestra.run.smithi094.stdout:5017 write 0xb3b8ab3 thru 0xb3c3ec7 (0xb415 bytes) 2024-09-21T22:51:33.953 INFO:teuthology.orchestra.run.smithi094.stdout:5018 trunc from 0xc4b210a to 0xd2b4c11 2024-09-21T22:51:33.966 INFO:teuthology.orchestra.run.smithi094.stdout:5019 read 0x60638d5 thru 0x606bf09 (0x8635 bytes) 2024-09-21T22:51:33.966 INFO:teuthology.orchestra.run.smithi094.stdout:5020 punch from 0x737ebd5 to 0x7388b84, (0x9faf bytes) 2024-09-21T22:51:33.966 INFO:teuthology.orchestra.run.smithi094.stdout:5023 write 0xa10b9ca thru 0xa11840e (0xca45 bytes) 2024-09-21T22:51:33.972 INFO:teuthology.orchestra.run.smithi094.stdout:5024 writesame 0xc92f5a4 thru 0xc93def1 data_size 0x1(0xe94e bytes) 2024-09-21T22:51:33.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:34.013 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:34.013 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:34.024 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:34.019+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:34.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:34.036 INFO:teuthology.orchestra.run.smithi094.stdout:5025 writesame 0x31324ca thru 0x3138e28 data_size 0x41(0x695f bytes) 2024-09-21T22:51:34.040 INFO:teuthology.orchestra.run.smithi094.stdout:5026 read 0x4c87861 thru 0x4c9483e (0xcfde bytes) 2024-09-21T22:51:34.041 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.041 INFO:teuthology.orchestra.run.smithi094.stdout:5028 punch from 0xb35b161 to 0xb366f43, (0xbde2 bytes) 2024-09-21T22:51:34.041 INFO:teuthology.orchestra.run.smithi094.stdout:5029 read 0xa215b2a thru 0xa21ddb4 (0x828b bytes) 2024-09-21T22:51:34.042 INFO:teuthology.orchestra.run.smithi094.stdout:5030 writesame 0x3f6df89 thru 0x3f7403b data_size 0x1(0x60b3 bytes) 2024-09-21T22:51:34.059 INFO:teuthology.orchestra.run.smithi094.stdout:5031 read 0xa57769d thru 0xa5839b0 (0xc314 bytes) 2024-09-21T22:51:34.060 INFO:teuthology.orchestra.run.smithi094.stdout:5032 write 0x5e1b183 thru 0x5e2620e (0xb08c bytes) 2024-09-21T22:51:34.063 INFO:teuthology.orchestra.run.smithi094.stdout:5034 write 0x8868edb thru 0x886c1db (0x3301 bytes) 2024-09-21T22:51:34.068 INFO:teuthology.orchestra.run.smithi094.stdout:5036 read 0x929d4c4 thru 0x929edb1 (0x18ee bytes) 2024-09-21T22:51:34.069 INFO:teuthology.orchestra.run.smithi094.stdout:5037 read 0x872f11a thru 0x873ab96 (0xba7d bytes) 2024-09-21T22:51:34.070 INFO:teuthology.orchestra.run.smithi094.stdout:5039 read 0xbab0fd9 thru 0xbab5095 (0x40bd bytes) 2024-09-21T22:51:34.070 INFO:teuthology.orchestra.run.smithi094.stdout:5042 write 0x180fc18 thru 0x181b317 (0xb700 bytes) 2024-09-21T22:51:34.078 INFO:teuthology.orchestra.run.smithi094.stdout:5043 read 0x19f3fb6 thru 0x19fb599 (0x75e4 bytes) 2024-09-21T22:51:34.078 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.079 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.079 INFO:teuthology.orchestra.run.smithi094.stdout:5048 write 0x4084513 thru 0x4085cd6 (0x17c4 bytes) 2024-09-21T22:51:34.082 INFO:teuthology.orchestra.run.smithi094.stdout:5049 trunc from 0xd2b4c11 to 0xc555c61 2024-09-21T22:51:34.096 INFO:teuthology.orchestra.run.smithi094.stdout:5054 read 0x98bd1b2 thru 0x98c6ebb (0x9d0a bytes) 2024-09-21T22:51:34.097 INFO:teuthology.orchestra.run.smithi094.stdout:5056 punch from 0x86ff7a5 to 0x8708beb, (0x9446 bytes) 2024-09-21T22:51:34.098 INFO:teuthology.orchestra.run.smithi094.stdout:5057 read 0x83611d3 thru 0x836c1ec (0xb01a bytes) 2024-09-21T22:51:34.098 INFO:teuthology.orchestra.run.smithi094.stdout:5058 write 0x4d94acb thru 0x4da44d5 (0xfa0b bytes) 2024-09-21T22:51:34.111 INFO:teuthology.orchestra.run.smithi094.stdout:5059 trunc from 0xc555c61 to 0x53a8a80 2024-09-21T22:51:34.118 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:34.118 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:34.124 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:34.119+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:34.126 INFO:teuthology.orchestra.run.smithi094.stdout:5060 punch from 0x324d1c1 to 0x324f911, (0x2750 bytes) 2024-09-21T22:51:34.126 INFO:teuthology.orchestra.run.smithi094.stdout:5061 punch from 0x141fb9c to 0x1421cec, (0x2150 bytes) 2024-09-21T22:51:34.126 INFO:teuthology.orchestra.run.smithi094.stdout:5062 read 0x145e839 thru 0x146e7c7 (0xff8f bytes) 2024-09-21T22:51:34.127 INFO:teuthology.orchestra.run.smithi094.stdout:5064 punch from 0xfe4662 to 0xfec5ff, (0x7f9d bytes) 2024-09-21T22:51:34.127 INFO:teuthology.orchestra.run.smithi094.stdout:5065 read 0x35ba644 thru 0x35c315e (0x8b1b bytes) 2024-09-21T22:51:34.135 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:34.135 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:34.143 INFO:teuthology.orchestra.run.smithi094.stdout:5066 writesame 0x8e9adfb thru 0x8e9e8a9 data_size 0x1(0x3aaf bytes) 2024-09-21T22:51:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:5067 read 0x2f3431c thru 0x2f40d37 (0xca1c bytes) 2024-09-21T22:51:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:5070 punch from 0x2eca375 to 0x2ecb17e, (0xe09 bytes) 2024-09-21T22:51:34.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:34.163 INFO:teuthology.orchestra.run.smithi094.stdout:5071 write 0xdf8af9f thru 0xdf99761 (0xe7c3 bytes) 2024-09-21T22:51:34.165 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.165 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:34.165 INFO:teuthology.orchestra.run.smithi094.stdout:5072 read 0x317771 thru 0x31d03b (0x58cb bytes) 2024-09-21T22:51:34.165 INFO:teuthology.orchestra.run.smithi094.stdout:5073 read 0x8a2b95a thru 0x8a36daa (0xb451 bytes) 2024-09-21T22:51:34.179 INFO:teuthology.orchestra.run.smithi094.stdout:5074 trunc from 0xdf99762 to 0x679d6a7 2024-09-21T22:51:34.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:34.221 INFO:teuthology.orchestra.run.smithi094.stdout:5075 writesame 0xe746c09 thru 0xe74bfbc data_size 0x16(0x53b4 bytes) 2024-09-21T22:51:34.225 INFO:teuthology.orchestra.run.smithi094.stdout:5076 read 0x6453706 thru 0x6457d72 (0x466d bytes) 2024-09-21T22:51:34.225 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:34.223+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:34.226 INFO:teuthology.orchestra.run.smithi094.stdout:5077 write 0x891cede thru 0x8923a79 (0x6b9c bytes) 2024-09-21T22:51:34.227 INFO:teuthology.orchestra.run.smithi094.stdout:5078 read 0x7b2ce26 thru 0x7b302b5 (0x3490 bytes) 2024-09-21T22:51:34.228 INFO:teuthology.orchestra.run.smithi094.stdout:5079 read 0xd42baf4 thru 0xd43b165 (0xf672 bytes) 2024-09-21T22:51:34.229 INFO:teuthology.orchestra.run.smithi094.stdout:5080 writesame 0x16cc172 thru 0x16dae79 data_size 0x1(0xed08 bytes) 2024-09-21T22:51:34.304 INFO:teuthology.orchestra.run.smithi094.stdout:5081 write 0xc6e7a05 thru 0xc6ec9b6 (0x4fb2 bytes) 2024-09-21T22:51:34.308 INFO:teuthology.orchestra.run.smithi094.stdout:5082 writesame 0xc13e114 thru 0xc148cb0 data_size 0x1(0xab9d bytes) 2024-09-21T22:51:34.326 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:34.323+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:34.339 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.339 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:34.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:34.354 INFO:teuthology.orchestra.run.smithi094.stdout:5083 punch from 0x27a8649 to 0x27ab3f3, (0x2daa bytes) 2024-09-21T22:51:34.354 INFO:teuthology.orchestra.run.smithi094.stdout:5084 write 0xdd1006c thru 0xdd1e599 (0xe52e bytes) 2024-09-21T22:51:34.357 INFO:teuthology.orchestra.run.smithi094.stdout:5085 write 0x78f827 thru 0x793c54 (0x442e bytes) 2024-09-21T22:51:34.360 INFO:teuthology.orchestra.run.smithi094.stdout:5086 writesame 0x1baabe7 thru 0x1bb42e5 data_size 0xf(0x96ff bytes) 2024-09-21T22:51:34.366 INFO:teuthology.orchestra.run.smithi094.stdout:5087 write 0x410af12 thru 0x411945e (0xe54d bytes) 2024-09-21T22:51:34.368 INFO:teuthology.orchestra.run.smithi094.stdout:5088 punch from 0x2aaf05b to 0x2ab86eb, (0x9690 bytes) 2024-09-21T22:51:34.368 INFO:teuthology.orchestra.run.smithi094.stdout:5089 read 0x7c86a0 thru 0x7d6027 (0xd988 bytes) 2024-09-21T22:51:34.378 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:34.378 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:34.383 INFO:teuthology.orchestra.run.smithi094.stdout:5091 trunc from 0xe74bfbd to 0x43cc0f7 2024-09-21T22:51:34.409 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:34.409 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:34.416 INFO:teuthology.orchestra.run.smithi094.stdout:5092 write 0x84d20de thru 0x84d4b12 (0x2a35 bytes) 2024-09-21T22:51:34.426 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:34.423+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:34.436 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:34.436 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:34.437 INFO:teuthology.orchestra.run.smithi094.stdout:5093 write 0xde332c5 thru 0xde42ab3 (0xf7ef bytes) 2024-09-21T22:51:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:5095 punch from 0x2957e6c to 0x29615f5, (0x9789 bytes) 2024-09-21T22:51:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:5100 write 0x82f20bd thru 0x8300d19 (0xec5d bytes) 2024-09-21T22:51:34.444 INFO:teuthology.orchestra.run.smithi094.stdout:5101 read 0xcbfbd0e thru 0xcc0501b (0x930e bytes) 2024-09-21T22:51:34.445 INFO:teuthology.orchestra.run.smithi094.stdout:5103 write 0xba60bd1 thru 0xba64523 (0x3953 bytes) 2024-09-21T22:51:34.449 INFO:teuthology.orchestra.run.smithi094.stdout:5105 read 0xd76c403 thru 0xd7781a9 (0xbda7 bytes) 2024-09-21T22:51:34.450 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.450 INFO:teuthology.orchestra.run.smithi094.stdout:5107 read 0xd5ebfc1 thru 0xd5ede61 (0x1ea1 bytes) 2024-09-21T22:51:34.450 INFO:teuthology.orchestra.run.smithi094.stdout:5108 read 0x381e370 thru 0x381edbd (0xa4e bytes) 2024-09-21T22:51:34.451 INFO:teuthology.orchestra.run.smithi094.stdout:5110 punch from 0x31cb189 to 0x31d6fbc, (0xbe33 bytes) 2024-09-21T22:51:34.452 INFO:teuthology.orchestra.run.smithi094.stdout:5112 punch from 0x69c46c9 to 0x69c8560, (0x3e97 bytes) 2024-09-21T22:51:34.452 INFO:teuthology.orchestra.run.smithi094.stdout:5114 writesame 0x87c8beb thru 0x87cd5a8 data_size 0x1(0x49be bytes) 2024-09-21T22:51:34.458 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:34.458 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:34.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:34.461 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:34.491 INFO:teuthology.orchestra.run.smithi094.stdout:5115 trunc from 0xde42ab4 to 0xa18f31 2024-09-21T22:51:34.492 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.492 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:34.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:34.527 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:34.523+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:34.550 INFO:teuthology.orchestra.run.smithi094.stdout:5117 trunc from 0xa18f31 to 0x7840006 2024-09-21T22:51:34.562 INFO:teuthology.orchestra.run.smithi094.stdout:5119 write 0x1f69887 thru 0x1f7832f (0xeaa9 bytes) 2024-09-21T22:51:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:5120 punch from 0x68297df to 0x682b321, (0x1b42 bytes) 2024-09-21T22:51:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:5122 punch from 0x4eef550 to 0x4efa958, (0xb408 bytes) 2024-09-21T22:51:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:5123 read 0x47e9d8 thru 0x48742b (0x8a54 bytes) 2024-09-21T22:51:34.570 INFO:teuthology.orchestra.run.smithi094.stdout:5125 read 0xba5e15 thru 0xbada3f (0x7c2b bytes) 2024-09-21T22:51:34.571 INFO:teuthology.orchestra.run.smithi094.stdout:5126 write 0xe6c174 thru 0xe7bd4a (0xfbd7 bytes) 2024-09-21T22:51:34.579 INFO:teuthology.orchestra.run.smithi094.stdout:5127 trunc from 0x7840006 to 0x5518c69 2024-09-21T22:51:34.595 INFO:teuthology.orchestra.run.smithi094.stdout:5128 write 0x3fac472 thru 0x3fb0cf9 (0x4888 bytes) 2024-09-21T22:51:34.601 INFO:teuthology.orchestra.run.smithi094.stdout:5129 read 0x25e4c29 thru 0x25e4cee (0xc6 bytes) 2024-09-21T22:51:34.615 INFO:teuthology.orchestra.run.smithi094.stdout:5130 write 0x61d3ec3 thru 0x61e0c20 (0xcd5e bytes) 2024-09-21T22:51:34.620 INFO:teuthology.orchestra.run.smithi094.stdout:5131 read 0x33a2955 thru 0x33a41ef (0x189b bytes) 2024-09-21T22:51:34.620 INFO:teuthology.orchestra.run.smithi094.stdout:5132 punch from 0x5e5434c to 0x5e5b460, (0x7114 bytes) 2024-09-21T22:51:34.627 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:34.620+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:34.638 INFO:teuthology.orchestra.run.smithi094.stdout:5133 write 0xa0264ea thru 0xa02cc3b (0x6752 bytes) 2024-09-21T22:51:34.645 INFO:teuthology.orchestra.run.smithi094.stdout:5134 read 0x7170418 thru 0x7170d37 (0x920 bytes) 2024-09-21T22:51:34.645 INFO:teuthology.orchestra.run.smithi094.stdout:5135 write 0x623b441 thru 0x623c576 (0x1136 bytes) 2024-09-21T22:51:34.648 INFO:teuthology.orchestra.run.smithi094.stdout:5136 write 0x1493381 thru 0x1498007 (0x4c87 bytes) 2024-09-21T22:51:34.652 INFO:teuthology.orchestra.run.smithi094.stdout:5137 write 0x5a1aa12 thru 0x5a1f11e (0x470d bytes) 2024-09-21T22:51:34.656 INFO:teuthology.orchestra.run.smithi094.stdout:5140 read 0x5e1cbc8 thru 0x5e2241f (0x5858 bytes) 2024-09-21T22:51:34.658 INFO:teuthology.orchestra.run.smithi094.stdout:5141 read 0x14e6c36 thru 0x14f10b5 (0xa480 bytes) 2024-09-21T22:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:5142 trunc from 0xa02cc3c to 0xe8bd77e 2024-09-21T22:51:34.671 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.671 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:34.682 INFO:teuthology.orchestra.run.smithi094.stdout:5143 read 0xe41ce82 thru 0xe422d17 (0x5e96 bytes) 2024-09-21T22:51:34.683 INFO:teuthology.orchestra.run.smithi094.stdout:5146 write 0x88515f2 thru 0x885a2b6 (0x8cc5 bytes) 2024-09-21T22:51:34.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:34.699 INFO:teuthology.orchestra.run.smithi094.stdout:5147 trunc from 0xe8bd77e to 0xae07126 2024-09-21T22:51:34.719 INFO:teuthology.orchestra.run.smithi094.stdout:5148 write 0x91f8e78 thru 0x9202adf (0x9c68 bytes) 2024-09-21T22:51:34.728 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:34.724+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:34.729 INFO:teuthology.orchestra.run.smithi094.stdout:5149 trunc from 0xae07126 to 0x6ae7ffb 2024-09-21T22:51:34.767 INFO:teuthology.orchestra.run.smithi094.stdout:5150 writesame 0xd01d046 thru 0xd023c6b data_size 0x2(0x6c26 bytes) 2024-09-21T22:51:34.769 INFO:teuthology.orchestra.run.smithi094.stdout:5151 trunc from 0xd023c6c to 0xd493412 2024-09-21T22:51:34.780 INFO:teuthology.orchestra.run.smithi094.stdout:5152 write 0xc130809 thru 0xc133fb8 (0x37b0 bytes) 2024-09-21T22:51:34.786 INFO:teuthology.orchestra.run.smithi094.stdout:5155 read 0x54c31ac thru 0x54c62aa (0x30ff bytes) 2024-09-21T22:51:34.787 INFO:teuthology.orchestra.run.smithi094.stdout:5157 writesame 0xd241714 thru 0xd2441b4 data_size 0x1(0x2aa1 bytes) 2024-09-21T22:51:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:5158 punch from 0x3160332 to 0x316ec38, (0xe906 bytes) 2024-09-21T22:51:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:5159 writesame 0xa8359a2 thru 0xa83e840 data_size 0x1(0x8e9f bytes) 2024-09-21T22:51:34.829 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:34.824+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:34.833 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.833 INFO:teuthology.orchestra.run.smithi094.stdout:5161 read 0x7168a18 thru 0x716e224 (0x580d bytes) 2024-09-21T22:51:34.834 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:34.836 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.836 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:34.843 INFO:teuthology.orchestra.run.smithi094.stdout:5163 trunc from 0xd493412 to 0x7a1dc93 2024-09-21T22:51:34.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:34.861 INFO:teuthology.orchestra.run.smithi094.stdout:5164 read 0x63e2bb4 thru 0x63e2cc4 (0x111 bytes) 2024-09-21T22:51:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:5165 trunc from 0x7a1dc93 to 0x16451f2 2024-09-21T22:51:34.906 INFO:teuthology.orchestra.run.smithi094.stdout:5167 punch from 0x10a1c36 to 0x10a8931, (0x6cfb bytes) 2024-09-21T22:51:34.927 INFO:teuthology.orchestra.run.smithi094.stdout:5168 write 0x6cf81c7 thru 0x6d02300 (0xa13a bytes) 2024-09-21T22:51:34.930 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:34.927+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:34.963 INFO:teuthology.orchestra.run.smithi094.stdout:5169 write 0xea9e44c thru 0xea9e87c (0x431 bytes) 2024-09-21T22:51:34.977 INFO:teuthology.orchestra.run.smithi094.stdout:5170 write 0xee13c9c thru 0xee234ab (0xf810 bytes) 2024-09-21T22:51:34.980 INFO:teuthology.orchestra.run.smithi094.stdout:5171 read 0xb90527 thru 0xb90636 (0x110 bytes) 2024-09-21T22:51:34.981 INFO:teuthology.orchestra.run.smithi094.stdout:5172 read 0xab2eae3 thru 0xab3998a (0xaea8 bytes) 2024-09-21T22:51:34.992 INFO:teuthology.orchestra.run.smithi094.stdout:5173 trunc from 0xee234ac to 0x7da497f 2024-09-21T22:51:34.993 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:34.994 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:35.010 INFO:teuthology.orchestra.run.smithi094.stdout:5174 write 0x4749403 thru 0x4753db6 (0xa9b4 bytes) 2024-09-21T22:51:35.031 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:35.027+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:35.032 INFO:teuthology.orchestra.run.smithi094.stdout:5176 write 0xaa98ff3 thru 0xaaa2d14 (0x9d22 bytes) 2024-09-21T22:51:35.039 INFO:teuthology.orchestra.run.smithi094.stdout:5177 write 0x41e3bbe thru 0x41ecf5e (0x93a1 bytes) 2024-09-21T22:51:35.041 INFO:teuthology.orchestra.run.smithi094.stdout:5178 write 0x9b35f48 thru 0x9b3fcde (0x9d97 bytes) 2024-09-21T22:51:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:5179 write 0xbe094b3 thru 0xbe10eaf (0x79fd bytes) 2024-09-21T22:51:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:5181 read 0x32618d thru 0x32d93e (0x77b2 bytes) 2024-09-21T22:51:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:5182 read 0x23883e thru 0x246945 (0xe108 bytes) 2024-09-21T22:51:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:5184 writesame 0x3778732 thru 0x3785867 data_size 0x3d(0xd136 bytes) 2024-09-21T22:51:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:5185 write 0x5e704dd thru 0x5e78077 (0x7b9b bytes) 2024-09-21T22:51:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:5186 read 0x6a85bd7 thru 0x6a95051 (0xf47b bytes) 2024-09-21T22:51:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:5188 write 0x153a579 thru 0x1542c91 (0x8719 bytes) 2024-09-21T22:51:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:5189 read 0x2d00f3f thru 0x2d07a8b (0x6b4d bytes) 2024-09-21T22:51:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:5192 writesame 0xdac8d44 thru 0xdac9e16 data_size 0x1(0x10d3 bytes) 2024-09-21T22:51:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:5193 read 0x5e44db2 thru 0x5e4f76b (0xa9ba bytes) 2024-09-21T22:51:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:5196 write 0x76f2ef0 thru 0x76f5e47 (0x2f58 bytes) 2024-09-21T22:51:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:5199 punch from 0x3277724 to 0x328730c, (0xfbe8 bytes) 2024-09-21T22:51:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:5200 punch from 0xd3a302b to 0xd3b0b69, (0xdb3e bytes) 2024-09-21T22:51:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:5201 trunc from 0xdac9e17 to 0x4e5e6fe 2024-09-21T22:51:35.132 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:35.127+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:5202 trunc from 0x4e5e6fe to 0xd30a29 2024-09-21T22:51:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:5203 punch from 0x1e9807 to 0x1ea795, (0xf8e bytes) 2024-09-21T22:51:35.194 INFO:teuthology.orchestra.run.smithi094.stdout:5204 write 0x1ad853f thru 0x1adc281 (0x3d43 bytes) 2024-09-21T22:51:35.200 INFO:teuthology.orchestra.run.smithi094.stdout:5205 punch from 0x15b71de to 0x15c6ccd, (0xfaef bytes) 2024-09-21T22:51:35.208 INFO:teuthology.orchestra.run.smithi094.stdout:5207 trunc from 0x1adc282 to 0x5661c7e 2024-09-21T22:51:35.233 INFO:teuthology.orchestra.run.smithi094.stdout:5208 write 0x8d1f465 thru 0x8d22b8f (0x372b bytes) 2024-09-21T22:51:35.233 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:35.231+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:35.238 INFO:teuthology.orchestra.run.smithi094.stdout:5210 read 0x4c28df7 thru 0x4c2fd25 (0x6f2f bytes) 2024-09-21T22:51:35.239 INFO:teuthology.orchestra.run.smithi094.stdout:5212 read 0x20b873 thru 0x213ed4 (0x8662 bytes) 2024-09-21T22:51:35.252 INFO:teuthology.orchestra.run.smithi094.stdout:5214 write 0x8de6c89 thru 0x8df17c1 (0xab39 bytes) 2024-09-21T22:51:35.255 INFO:teuthology.orchestra.run.smithi094.stdout:5215 punch from 0x7ab0070 to 0x7ab17cb, (0x175b bytes) 2024-09-21T22:51:35.268 INFO:teuthology.orchestra.run.smithi094.stdout:5216 write 0x99b1fc7 thru 0x99bf74d (0xd787 bytes) 2024-09-21T22:51:35.272 INFO:teuthology.orchestra.run.smithi094.stdout:5217 punch from 0x3e6e7de to 0x3e7d716, (0xef38 bytes) 2024-09-21T22:51:35.272 INFO:teuthology.orchestra.run.smithi094.stdout:5218 write 0x51c51e1 thru 0x51d2fbb (0xdddb bytes) 2024-09-21T22:51:35.276 INFO:teuthology.orchestra.run.smithi094.stdout:5219 write 0x101f0f8 thru 0x1022545 (0x344e bytes) 2024-09-21T22:51:35.306 INFO:teuthology.orchestra.run.smithi094.stdout:5220 write 0xd93a924 thru 0xd9488b9 (0xdf96 bytes) 2024-09-21T22:51:35.309 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:35.309 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.311 INFO:teuthology.orchestra.run.smithi094.stdout:5221 read 0xc732a28 thru 0xc7404f2 (0xdacb bytes) 2024-09-21T22:51:35.311 INFO:teuthology.orchestra.run.smithi094.stdout:5222 read 0xca533da thru 0xca5ffbd (0xcbe4 bytes) 2024-09-21T22:51:35.311 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.311 INFO:teuthology.orchestra.run.smithi094.stdout:5229 write 0xc55e5ce thru 0xc561cb2 (0x36e5 bytes) 2024-09-21T22:51:35.317 INFO:teuthology.orchestra.run.smithi094.stdout:5230 write 0xb52211b thru 0xb5246d5 (0x25bb bytes) 2024-09-21T22:51:35.324 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:35.325 INFO:teuthology.orchestra.run.smithi094.stdout:5231 read 0xc9e22a5 thru 0xc9f00f8 (0xde54 bytes) 2024-09-21T22:51:35.325 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.325 INFO:teuthology.orchestra.run.smithi094.stdout:5233 punch from 0x6d1424b to 0x6d2188b, (0xd640 bytes) 2024-09-21T22:51:35.326 INFO:teuthology.orchestra.run.smithi094.stdout:5235 trunc from 0xd9488ba to 0xd1dd438 2024-09-21T22:51:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:5237 read 0x2048a72 thru 0x204b502 (0x2a91 bytes) 2024-09-21T22:51:35.344 INFO:teuthology.orchestra.run.smithi094.stdout:5238 writesame 0x656d80d thru 0x657b775 data_size 0x1(0xdf69 bytes) 2024-09-21T22:51:35.370 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:35.367+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:35.413 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:35.408+0000 7fb1fe3537c0 -1 Falling back to public interface 2024-09-21T22:51:35.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:51:35.430 INFO:teuthology.orchestra.run.smithi094.stdout:5239 trunc from 0xd1dd438 to 0x669e87a 2024-09-21T22:51:35.434 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:35.431+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:35.445 INFO:teuthology.orchestra.run.smithi094.stdout:5240 trunc from 0x669e87a to 0x7965dd3 2024-09-21T22:51:35.462 INFO:teuthology.orchestra.run.smithi094.stdout:5241 trunc from 0x7965dd3 to 0xb57e065 2024-09-21T22:51:35.484 INFO:teuthology.orchestra.run.smithi094.stdout:5242 write 0xcced9dc thru 0xccedf9e (0x5c3 bytes) 2024-09-21T22:51:35.489 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:35.489 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.490 INFO:teuthology.orchestra.run.smithi094.stdout:5243 write 0x41958cc thru 0x419e3fc (0x8b31 bytes) 2024-09-21T22:51:35.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:35.505 INFO:teuthology.orchestra.run.smithi094.stdout:5244 write 0xee362a5 thru 0xee3f04e (0x8daa bytes) 2024-09-21T22:51:35.508 INFO:teuthology.orchestra.run.smithi094.stdout:5245 write 0xed399ce thru 0xed424b7 (0x8aea bytes) 2024-09-21T22:51:35.527 INFO:teuthology.orchestra.run.smithi094.stdout:5246 trunc from 0xee3f04f to 0x5a6eeb4 2024-09-21T22:51:35.546 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:35.543+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:35.550 INFO:teuthology.orchestra.run.smithi094.stdout:5247 trunc from 0x5a6eeb4 to 0xb5ffdfd 2024-09-21T22:51:35.574 INFO:teuthology.orchestra.run.smithi094.stdout:5248 trunc from 0xb5ffdfd to 0x1766e01 2024-09-21T22:51:35.594 INFO:teuthology.orchestra.run.smithi094.stdout:5251 read 0x11fe8d6 thru 0x1208c6d (0xa398 bytes) 2024-09-21T22:51:35.595 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.595 INFO:teuthology.orchestra.run.smithi094.stdout:5253 read 0xa3ab2 thru 0xb19cf (0xdf1e bytes) 2024-09-21T22:51:35.621 INFO:teuthology.orchestra.run.smithi094.stdout:5254 write 0xbb11650 thru 0xbb1c668 (0xb019 bytes) 2024-09-21T22:51:35.626 INFO:teuthology.orchestra.run.smithi094.stdout:5255 write 0x78822c8 thru 0x788d254 (0xaf8d bytes) 2024-09-21T22:51:35.635 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:35.631+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:35.645 INFO:teuthology.orchestra.run.smithi094.stdout:5256 write 0xec75207 thru 0xec75971 (0x76b bytes) 2024-09-21T22:51:35.648 INFO:teuthology.orchestra.run.smithi094.stdout:5257 write 0xe4b90aa thru 0xe4b9b85 (0xadc bytes) 2024-09-21T22:51:35.651 INFO:teuthology.orchestra.run.smithi094.stdout:5258 write 0x40976fc thru 0x40a4f09 (0xd80e bytes) 2024-09-21T22:51:35.657 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.657 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.657 INFO:teuthology.orchestra.run.smithi094.stdout:5262 flatten 2024-09-21T22:51:35.666 INFO:teuthology.orchestra.run.smithi094.stdout:5263 read 0x87c5d99 thru 0x87cdf8d (0x81f5 bytes) 2024-09-21T22:51:35.666 INFO:teuthology.orchestra.run.smithi094.stdout:5265 read 0xb945931 thru 0xb948b23 (0x31f3 bytes) 2024-09-21T22:51:35.667 INFO:teuthology.orchestra.run.smithi094.stdout:5266 read 0x58214e7 thru 0x582dca4 (0xc7be bytes) 2024-09-21T22:51:35.668 INFO:teuthology.orchestra.run.smithi094.stdout:5267 write 0xd7f83e8 thru 0xd800a6d (0x8686 bytes) 2024-09-21T22:51:35.672 INFO:teuthology.orchestra.run.smithi094.stdout:5269 read 0x2eab06e thru 0x2eafd1b (0x4cae bytes) 2024-09-21T22:51:35.673 INFO:teuthology.orchestra.run.smithi094.stdout:5271 write 0x4f80876 thru 0x4f85451 (0x4bdc bytes) 2024-09-21T22:51:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:35.695 INFO:teuthology.orchestra.run.smithi094.stdout:5272 trunc from 0xec75972 to 0x2c53f51 2024-09-21T22:51:35.714 INFO:teuthology.orchestra.run.smithi094.stdout:5273 punch from 0x5b42bc to 0x5bed85, (0xaac9 bytes) 2024-09-21T22:51:35.737 INFO:teuthology.orchestra.run.smithi094.stdout:5276 write 0xc409eeb thru 0xc41240d (0x8523 bytes) 2024-09-21T22:51:35.738 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:35.735+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:5278 punch from 0x203b343 to 0x2049d84, (0xea41 bytes) 2024-09-21T22:51:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:5282 read 0x154b586 thru 0x155137e (0x5df9 bytes) 2024-09-21T22:51:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:5283 read 0x7d9ecbb thru 0x7da8ed5 (0xa21b bytes) 2024-09-21T22:51:35.742 INFO:teuthology.orchestra.run.smithi094.stdout:5284 write 0x3782b3c thru 0x378ca6a (0x9f2f bytes) 2024-09-21T22:51:35.745 INFO:teuthology.orchestra.run.smithi094.stdout:5285 read 0x3ea7be1 thru 0x3ea7e4c (0x26c bytes) 2024-09-21T22:51:35.745 INFO:teuthology.orchestra.run.smithi094.stdout:5287 write 0x6f5971a thru 0x6f5ef4f (0x5836 bytes) 2024-09-21T22:51:35.750 INFO:teuthology.orchestra.run.smithi094.stdout:5288 read 0xa9f1aa5 thru 0xa9f8f89 (0x74e5 bytes) 2024-09-21T22:51:35.750 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.751 INFO:teuthology.orchestra.run.smithi094.stdout:5290 write 0x499bddd thru 0x49a9799 (0xd9bd bytes) 2024-09-21T22:51:35.753 INFO:teuthology.orchestra.run.smithi094.stdout:5291 write 0x2ef822d thru 0x2efca9e (0x4872 bytes) 2024-09-21T22:51:35.755 INFO:teuthology.orchestra.run.smithi094.stdout:5294 write 0x745be78 thru 0x746a924 (0xeaad bytes) 2024-09-21T22:51:35.757 INFO:teuthology.orchestra.run.smithi094.stdout:5295 read 0x463a068 thru 0x463f57c (0x5515 bytes) 2024-09-21T22:51:35.759 INFO:teuthology.orchestra.run.smithi094.stdout:5296 trunc from 0xc41240e to 0xd69c49d 2024-09-21T22:51:35.767 INFO:teuthology.orchestra.run.smithi094.stdout:5299 writesame 0x2f625f9 thru 0x2f700d1 data_size 0x53(0xdad9 bytes) 2024-09-21T22:51:35.769 INFO:teuthology.orchestra.run.smithi094.stdout:5300 punch from 0x3e684e8 to 0x3e6a0bc, (0x1bd4 bytes) 2024-09-21T22:51:35.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:35.792 INFO:teuthology.orchestra.run.smithi094.stdout:5304 trunc from 0xd69c49d to 0x3e4207 2024-09-21T22:51:35.837 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:35.831+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:35.838 INFO:teuthology.orchestra.run.smithi094.stdout:5305 write 0x5c1e97c thru 0x5c2b756 (0xcddb bytes) 2024-09-21T22:51:35.844 INFO:teuthology.orchestra.run.smithi094.stdout:5306 write 0x77acfb thru 0x77d453 (0x2759 bytes) 2024-09-21T22:51:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:5307 punch from 0x711085 to 0x714d61, (0x3cdc bytes) 2024-09-21T22:51:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:5309 read 0x4b78622 thru 0x4b7f925 (0x7304 bytes) 2024-09-21T22:51:35.848 INFO:teuthology.orchestra.run.smithi094.stdout:5310 read 0xcb443a thru 0xcc2b6c (0xe733 bytes) 2024-09-21T22:51:35.861 INFO:teuthology.orchestra.run.smithi094.stdout:5313 trunc from 0x5c2b757 to 0xb99bbf9 2024-09-21T22:51:35.863 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:35.864 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:35.870 INFO:teuthology.orchestra.run.smithi094.stdout:5316 writesame 0x23ce38c thru 0x23d179e data_size 0x1(0x3413 bytes) 2024-09-21T22:51:35.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:35.890 INFO:teuthology.orchestra.run.smithi094.stdout:5319 writesame 0x9061b14 thru 0x907118d data_size 0x7(0xf67a bytes) 2024-09-21T22:51:35.902 INFO:teuthology.orchestra.run.smithi094.stdout:5320 read 0xa1ea4de thru 0xa1f8546 (0xe069 bytes) 2024-09-21T22:51:35.904 INFO:teuthology.orchestra.run.smithi094.stdout:5321 read 0xf05450 thru 0xf0af3f (0x5af0 bytes) 2024-09-21T22:51:35.904 INFO:teuthology.orchestra.run.smithi094.stdout:5324 write 0x5ec160a thru 0x5ec8c1d (0x7614 bytes) 2024-09-21T22:51:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:5326 trunc from 0xb99bbf9 to 0xd149c08 2024-09-21T22:51:35.924 INFO:teuthology.orchestra.run.smithi094.stdout:5327 read 0xb15aac1 thru 0xb15cfc1 (0x2501 bytes) 2024-09-21T22:51:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:5329 punch from 0xe3047c to 0xe381c1, (0x7d45 bytes) 2024-09-21T22:51:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:5330 write 0xc226d6f thru 0xc232ffe (0xc290 bytes) 2024-09-21T22:51:35.931 INFO:teuthology.orchestra.run.smithi094.stdout:5331 punch from 0xbec09c7 to 0xbecdc7b, (0xd2b4 bytes) 2024-09-21T22:51:35.931 INFO:teuthology.orchestra.run.smithi094.stdout:5333 write 0x7242a1d thru 0x7243588 (0xb6c bytes) 2024-09-21T22:51:35.937 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:35.932+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:35.961 INFO:teuthology.orchestra.run.smithi094.stdout:5334 trunc from 0xd149c08 to 0x343ad9c 2024-09-21T22:51:35.974 INFO:teuthology.orchestra.run.smithi094.stdout:5335 read 0x15e1212 thru 0x15e9356 (0x8145 bytes) 2024-09-21T22:51:35.975 INFO:teuthology.orchestra.run.smithi094.stdout:5336 writesame 0x6c8d1d thru 0x6d406c data_size 0x97(0xb350 bytes) 2024-09-21T22:51:35.982 INFO:teuthology.orchestra.run.smithi094.stdout:5337 trunc from 0x343ad9c to 0x4d73a6f 2024-09-21T22:51:35.992 INFO:teuthology.orchestra.run.smithi094.stdout:5339 punch from 0x4390a1d to 0x439a6ea, (0x9ccd bytes) 2024-09-21T22:51:36.008 INFO:teuthology.orchestra.run.smithi094.stdout:5342 write 0x8e9bf7f thru 0x8ea18bd (0x593f bytes) 2024-09-21T22:51:36.014 INFO:teuthology.orchestra.run.smithi094.stdout:5343 read 0x5f0ae24 thru 0x5f13fb6 (0x9193 bytes) 2024-09-21T22:51:36.014 INFO:teuthology.orchestra.run.smithi094.stdout:5345 read 0x73b18eb thru 0x73b3bf5 (0x230b bytes) 2024-09-21T22:51:36.015 INFO:teuthology.orchestra.run.smithi094.stdout:5347 read 0x6311429 thru 0x6318f7b (0x7b53 bytes) 2024-09-21T22:51:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:5349 write 0xe88fa07 thru 0xe892a83 (0x307d bytes) 2024-09-21T22:51:36.038 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:36.032+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:36.038 INFO:teuthology.orchestra.run.smithi094.stdout:5352 read 0xda09974 thru 0xda15ff4 (0xc681 bytes) 2024-09-21T22:51:36.039 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.046 INFO:teuthology.orchestra.run.smithi094.stdout:5354 trunc from 0xe892a84 to 0xb261a22 2024-09-21T22:51:36.051 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:36.051 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:36.062 INFO:teuthology.orchestra.run.smithi094.stdout:5357 read 0x5d28429 thru 0x5d2c78c (0x4364 bytes) 2024-09-21T22:51:36.063 INFO:teuthology.orchestra.run.smithi094.stdout:5358 read 0x92e8f7d thru 0x92f6429 (0xd4ad bytes) 2024-09-21T22:51:36.064 INFO:teuthology.orchestra.run.smithi094.stdout:5360 write 0x22d5767 thru 0x22d8032 (0x28cc bytes) 2024-09-21T22:51:36.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:36.067 INFO:teuthology.orchestra.run.smithi094.stdout:5362 read 0xaebe1cb thru 0xaec89ad (0xa7e3 bytes) 2024-09-21T22:51:36.067 INFO:teuthology.orchestra.run.smithi094.stdout:5363 punch from 0x36ea3f5 to 0x36edf93, (0x3b9e bytes) 2024-09-21T22:51:36.067 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.067 INFO:teuthology.orchestra.run.smithi094.stdout:5365 writesame 0x91a20ce thru 0x91b1925 data_size 0x12(0xf858 bytes) 2024-09-21T22:51:36.088 INFO:teuthology.orchestra.run.smithi094.stdout:5366 trunc from 0xb261a22 to 0x5ba4d64 2024-09-21T22:51:36.137 INFO:teuthology.orchestra.run.smithi094.stdout:5367 write 0xcef484e thru 0xcf04175 (0xf928 bytes) 2024-09-21T22:51:36.138 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:36.132+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:36.144 INFO:teuthology.orchestra.run.smithi094.stdout:5368 read 0x1a52efa thru 0x1a5d60d (0xa714 bytes) 2024-09-21T22:51:36.144 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.144 INFO:teuthology.orchestra.run.smithi094.stdout:5370 read 0x6f69fcf thru 0x6f70670 (0x66a2 bytes) 2024-09-21T22:51:36.145 INFO:teuthology.orchestra.run.smithi094.stdout:5371 read 0x67f0674 thru 0x67f48cd (0x425a bytes) 2024-09-21T22:51:36.145 INFO:teuthology.orchestra.run.smithi094.stdout:5372 punch from 0x81e767b to 0x81eee99, (0x781e bytes) 2024-09-21T22:51:36.145 INFO:teuthology.orchestra.run.smithi094.stdout:5373 read 0x1fa788b thru 0x1faf2b7 (0x7a2d bytes) 2024-09-21T22:51:36.145 INFO:teuthology.orchestra.run.smithi094.stdout:5374 read 0x2b29bba thru 0x2b2b24c (0x1693 bytes) 2024-09-21T22:51:36.145 INFO:teuthology.orchestra.run.smithi094.stdout:5375 read 0x8b90ec thru 0x8bdcb4 (0x4bc9 bytes) 2024-09-21T22:51:36.150 INFO:teuthology.orchestra.run.smithi094.stdout:5376 trunc from 0xcf04176 to 0xece5d66 2024-09-21T22:51:36.164 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.164 INFO:teuthology.orchestra.run.smithi094.stdout:5378 read 0xcf86fb5 thru 0xcf92918 (0xb964 bytes) 2024-09-21T22:51:36.165 INFO:teuthology.orchestra.run.smithi094.stdout:5379 write 0x7bb2004 thru 0x7bc1cf5 (0xfcf2 bytes) 2024-09-21T22:51:36.168 INFO:teuthology.orchestra.run.smithi094.stdout:5381 write 0xac35902 thru 0xac40dbc (0xb4bb bytes) 2024-09-21T22:51:36.176 INFO:teuthology.orchestra.run.smithi094.stdout:5383 writesame 0xcdf15f5 thru 0xce014e0 data_size 0x82(0xfeec bytes) 2024-09-21T22:51:36.180 INFO:teuthology.orchestra.run.smithi094.stdout:5385 writesame 0xacfc4ba thru 0xad03d3e data_size 0x1(0x7885 bytes) 2024-09-21T22:51:36.191 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:36.191 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:36.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:36.220 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.220 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.221 INFO:teuthology.orchestra.run.smithi094.stdout:5391 punch from 0x576f915 to 0x5774892, (0x4f7d bytes) 2024-09-21T22:51:36.221 INFO:teuthology.orchestra.run.smithi094.stdout:5392 read 0x289b9bb thru 0x28a38cc (0x7f12 bytes) 2024-09-21T22:51:36.222 INFO:teuthology.orchestra.run.smithi094.stdout:5393 write 0xc89cf39 thru 0xc8a0028 (0x30f0 bytes) 2024-09-21T22:51:36.224 INFO:teuthology.orchestra.run.smithi094.stdout:5396 read 0x127cb0e thru 0x127faef (0x2fe2 bytes) 2024-09-21T22:51:36.225 INFO:teuthology.orchestra.run.smithi094.stdout:5397 write 0xe37656d thru 0xe376fbf (0xa53 bytes) 2024-09-21T22:51:36.231 INFO:teuthology.orchestra.run.smithi094.stdout:5398 write 0x9850de6 thru 0x985d3aa (0xc5c5 bytes) 2024-09-21T22:51:36.234 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.234 INFO:teuthology.orchestra.run.smithi094.stdout:5403 read 0x343cef3 thru 0x343d5af (0x6bd bytes) 2024-09-21T22:51:36.235 INFO:teuthology.orchestra.run.smithi094.stdout:5404 punch from 0x4b532db to 0x4b56835, (0x355a bytes) 2024-09-21T22:51:36.235 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.253 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:36.251+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:36.261 INFO:teuthology.orchestra.run.smithi094.stdout:5408 trunc from 0xece5d66 to 0x4ba9718 2024-09-21T22:51:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:5410 read 0x2f1ebcd thru 0x2f20c8e (0x20c2 bytes) 2024-09-21T22:51:36.279 INFO:teuthology.orchestra.run.smithi094.stdout:5411 write 0x359f9bf thru 0x35ac7f2 (0xce34 bytes) 2024-09-21T22:51:36.294 INFO:teuthology.orchestra.run.smithi094.stdout:5412 writesame 0x63b0ec0 thru 0x63b4a33 data_size 0xa(0x3b74 bytes) 2024-09-21T22:51:36.296 INFO:teuthology.orchestra.run.smithi094.stdout:5413 punch from 0x201d42d to 0x202b1a8, (0xdd7b bytes) 2024-09-21T22:51:36.296 INFO:teuthology.orchestra.run.smithi094.stdout:5414 punch from 0x3b549b0 to 0x3b55907, (0xf57 bytes) 2024-09-21T22:51:36.311 INFO:teuthology.orchestra.run.smithi094.stdout:5415 trunc from 0x63b4a34 to 0xde84f0a 2024-09-21T22:51:36.324 INFO:teuthology.orchestra.run.smithi094.stdout:5416 read 0x4ece230 thru 0x4ed2b8f (0x4960 bytes) 2024-09-21T22:51:36.324 INFO:teuthology.orchestra.run.smithi094.stdout:5418 write 0xd044d70 thru 0xd04a5d2 (0x5863 bytes) 2024-09-21T22:51:36.327 INFO:teuthology.orchestra.run.smithi094.stdout:5419 punch from 0xa8afb83 to 0xa8bc4a6, (0xc923 bytes) 2024-09-21T22:51:36.334 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:36.335 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:36.338 INFO:teuthology.orchestra.run.smithi094.stdout:5420 writesame 0xe08ff5d thru 0xe0930f0 data_size 0x26(0x3194 bytes) 2024-09-21T22:51:36.339 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:36.335+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:36.342 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.342 INFO:teuthology.orchestra.run.smithi094.stdout:5423 read 0x79da530 thru 0x79e3c7a (0x974b bytes) 2024-09-21T22:51:36.343 INFO:teuthology.orchestra.run.smithi094.stdout:5425 punch from 0x171955a to 0x171a3b0, (0xe56 bytes) 2024-09-21T22:51:36.343 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.343 INFO:teuthology.orchestra.run.smithi094.stdout:5430 write 0x752fa96 thru 0x75383c7 (0x8932 bytes) 2024-09-21T22:51:36.346 INFO:teuthology.orchestra.run.smithi094.stdout:5431 read 0x55dbe1e thru 0x55de709 (0x28ec bytes) 2024-09-21T22:51:36.347 INFO:teuthology.orchestra.run.smithi094.stdout:5432 write 0x5f2cb79 thru 0x5f3c99f (0xfe27 bytes) 2024-09-21T22:51:36.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:36.351 INFO:teuthology.orchestra.run.smithi094.stdout:5433 read 0xa8b00fd thru 0xa8bc797 (0xc69b bytes) 2024-09-21T22:51:36.370 INFO:teuthology.orchestra.run.smithi094.stdout:5435 trunc from 0xe0930f1 to 0x6a63bd3 2024-09-21T22:51:36.384 INFO:teuthology.orchestra.run.smithi094.stdout:5436 read 0x27fb46f thru 0x2802bcf (0x7761 bytes) 2024-09-21T22:51:36.401 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:51:36.401 INFO:teuthology.orchestra.run.smithi094.stdout:5437 trunc from 0x6a63bd3 to 0xddce969 2024-09-21T22:51:36.415 INFO:teuthology.orchestra.run.smithi094.stdout:5439 write 0x159b6ad thru 0x159d553 (0x1ea7 bytes) 2024-09-21T22:51:36.417 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.418 INFO:teuthology.orchestra.run.smithi094.stdout:5442 write 0x676b2c4 thru 0x67753ff (0xa13c bytes) 2024-09-21T22:51:36.423 INFO:teuthology.orchestra.run.smithi094.stdout:5443 write 0x9791a4 thru 0x9801c3 (0x7020 bytes) 2024-09-21T22:51:36.426 INFO:teuthology.orchestra.run.smithi094.stdout:5444 write 0xb5b71e6 thru 0xb5c56a5 (0xe4c0 bytes) 2024-09-21T22:51:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:5445 write 0xe139419 thru 0xe145330 (0xbf18 bytes) 2024-09-21T22:51:36.440 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:36.436+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:36.441 INFO:teuthology.orchestra.run.smithi094.stdout:5447 read 0x3e268a thru 0x3eac2c (0x85a3 bytes) 2024-09-21T22:51:36.441 INFO:teuthology.orchestra.run.smithi094.stdout:5448 write 0xa792c08 thru 0xa7a1100 (0xe4f9 bytes) 2024-09-21T22:51:36.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:36.469 INFO:teuthology.orchestra.run.smithi094.stdout:5449 trunc from 0xe145331 to 0xf4e0ea 2024-09-21T22:51:36.508 INFO:teuthology.orchestra.run.smithi094.stdout:5450 trunc from 0xf4e0ea to 0x16f570a 2024-09-21T22:51:36.529 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:36.529 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:36.541 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:36.536+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:36.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:36.552 INFO:teuthology.orchestra.run.smithi094.stdout:5451 write 0xce08b7f thru 0xce13f2d (0xb3af bytes) 2024-09-21T22:51:36.557 INFO:teuthology.orchestra.run.smithi094.stdout:5453 punch from 0x93c44a1 to 0x93c590f, (0x146e bytes) 2024-09-21T22:51:36.558 INFO:teuthology.orchestra.run.smithi094.stdout:5454 punch from 0x8ce12b7 to 0x8ce15e3, (0x32c bytes) 2024-09-21T22:51:36.570 INFO:teuthology.orchestra.run.smithi094.stdout:5455 write 0xe68576c thru 0xe688cb0 (0x3545 bytes) 2024-09-21T22:51:36.572 INFO:teuthology.orchestra.run.smithi094.stdout:5456 write 0x94db959 thru 0x94e657f (0xac27 bytes) 2024-09-21T22:51:36.575 INFO:teuthology.orchestra.run.smithi094.stdout:5458 writesame 0x17ac6f0 thru 0x17b98c1 data_size 0x6b(0xd1d2 bytes) 2024-09-21T22:51:36.581 INFO:teuthology.orchestra.run.smithi094.stdout:5459 writesame 0x9763694 thru 0x9771159 data_size 0x29(0xdac6 bytes) 2024-09-21T22:51:36.585 INFO:teuthology.orchestra.run.smithi094.stdout:5461 writesame 0xdf254af thru 0xdf338fd data_size 0x1(0xe44f bytes) 2024-09-21T22:51:36.623 INFO:teuthology.orchestra.run.smithi094.stdout:5463 write 0x20384f4 thru 0x2045d27 (0xd834 bytes) 2024-09-21T22:51:36.627 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.628 INFO:teuthology.orchestra.run.smithi094.stdout:5467 write 0x6a99928 thru 0x6aa0f70 (0x7649 bytes) 2024-09-21T22:51:36.629 INFO:teuthology.orchestra.run.smithi094.stdout:5468 read 0xcfacd5b thru 0xcfb07eb (0x3a91 bytes) 2024-09-21T22:51:36.629 INFO:teuthology.orchestra.run.smithi094.stdout:5469 read 0xba0b861 thru 0xba1b53f (0xfcdf bytes) 2024-09-21T22:51:36.630 INFO:teuthology.orchestra.run.smithi094.stdout:5470 write 0xe518cd6 thru 0xe51966b (0x996 bytes) 2024-09-21T22:51:36.630 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:36.630 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:36.633 INFO:teuthology.orchestra.run.smithi094.stdout:5471 writesame 0xaec371e thru 0xaec3b08 data_size 0x1(0x3eb bytes) 2024-09-21T22:51:36.637 INFO:teuthology.orchestra.run.smithi094.stdout:5472 read 0xdc0c620 thru 0xdc0f7fe (0x31df bytes) 2024-09-21T22:51:36.638 INFO:teuthology.orchestra.run.smithi094.stdout:5473 write 0xb27d595 thru 0xb281e71 (0x48dd bytes) 2024-09-21T22:51:36.640 INFO:teuthology.orchestra.run.smithi094.stdout:5474 writesame 0x94336ff thru 0x9439375 data_size 0x1(0x5c77 bytes) 2024-09-21T22:51:36.642 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:36.639+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:36.648 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:36.648 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:36.653 INFO:teuthology.orchestra.run.smithi094.stdout:5475 write 0xb3cb66f thru 0xb3d78a4 (0xc236 bytes) 2024-09-21T22:51:36.655 INFO:teuthology.orchestra.run.smithi094.stdout:5478 read 0x20f7c8c thru 0x21041b3 (0xc528 bytes) 2024-09-21T22:51:36.655 INFO:teuthology.orchestra.run.smithi094.stdout:5479 write 0x2fa2a14 thru 0x2fa832a (0x5917 bytes) 2024-09-21T22:51:36.660 INFO:teuthology.orchestra.run.smithi094.stdout:5480 write 0x562ea41 thru 0x5632909 (0x3ec9 bytes) 2024-09-21T22:51:36.664 INFO:teuthology.orchestra.run.smithi094.stdout:5483 punch from 0xc5950de to 0xc5a1ac5, (0xc9e7 bytes) 2024-09-21T22:51:36.664 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.664 INFO:teuthology.orchestra.run.smithi094.stdout:5485 read 0xdd837e thru 0xde14c2 (0x9145 bytes) 2024-09-21T22:51:36.664 INFO:teuthology.orchestra.run.smithi094.stdout:5486 punch from 0x5e0d0f7 to 0x5e11779, (0x4682 bytes) 2024-09-21T22:51:36.664 INFO:teuthology.orchestra.run.smithi094.stdout:5487 write 0x4faf559 thru 0x4fba3e7 (0xae8f bytes) 2024-09-21T22:51:36.665 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:36.665 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:36.665 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:36.668 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:36.668 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:36.670 INFO:teuthology.orchestra.run.smithi094.stdout:5488 trunc from 0xe688cb1 to 0xb812a1c 2024-09-21T22:51:36.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:36.687 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:36.687 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:36.687 INFO:teuthology.orchestra.run.smithi094.stdout:5489 read 0x740a1f9 thru 0x740f3e4 (0x51ec bytes) 2024-09-21T22:51:36.688 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.688 INFO:teuthology.orchestra.run.smithi094.stdout:5491 writesame 0x2806bb4 thru 0x2812b46 data_size 0x1(0xbf93 bytes) 2024-09-21T22:51:36.706 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.706 INFO:teuthology.orchestra.run.smithi094.stdout:5493 read 0x6c32664 thru 0x6c39e45 (0x77e2 bytes) 2024-09-21T22:51:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:36.707 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:36.720 INFO:teuthology.orchestra.run.smithi094.stdout:5494 write 0xd471140 thru 0xd47c0c9 (0xaf8a bytes) 2024-09-21T22:51:36.724 INFO:teuthology.orchestra.run.smithi094.stdout:5495 write 0x3d4c18b thru 0x3d4ec4f (0x2ac5 bytes) 2024-09-21T22:51:36.725 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:36.728 INFO:teuthology.orchestra.run.smithi094.stdout:5496 read 0xb9d2383 thru 0xb9d524d (0x2ecb bytes) 2024-09-21T22:51:36.728 INFO:teuthology.orchestra.run.smithi094.stdout:5498 read 0x89ad7e2 thru 0x89b58aa (0x80c9 bytes) 2024-09-21T22:51:36.728 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.729 INFO:teuthology.orchestra.run.smithi094.stdout:5500 punch from 0x55ac672 to 0x55b353d, (0x6ecb bytes) 2024-09-21T22:51:36.736 INFO:teuthology.orchestra.run.smithi094.stdout:5501 trunc from 0xd47c0ca to 0x850777f 2024-09-21T22:51:36.746 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:36.746 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:36.749 INFO:teuthology.orchestra.run.smithi094.stdout:5502 read 0x4533902 thru 0x4534183 (0x882 bytes) 2024-09-21T22:51:36.749 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:36.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:36.750 INFO:teuthology.orchestra.run.smithi094.stdout:5503 punch from 0x6b813d to 0x6bdff4, (0x5eb7 bytes) 2024-09-21T22:51:36.750 INFO:teuthology.orchestra.run.smithi094.stdout:5505 write 0x1e6dadf thru 0x1e7c81a (0xed3c bytes) 2024-09-21T22:51:36.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.752 INFO:teuthology.orchestra.run.smithi094.stdout:5507 punch from 0x710fc8b to 0x71158b8, (0x5c2d bytes) 2024-09-21T22:51:36.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.759 INFO:teuthology.orchestra.run.smithi094.stdout:5511 trunc from 0x850777f to 0x2dcf63e 2024-09-21T22:51:36.773 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:36.771+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:5512 trunc from 0x2dcf63e to 0xcca7ef0 2024-09-21T22:51:36.797 INFO:teuthology.orchestra.run.smithi094.stdout:5513 punch from 0x80a8ee1 to 0x80acb89, (0x3ca8 bytes) 2024-09-21T22:51:36.797 INFO:teuthology.orchestra.run.smithi094.stdout:5514 write 0x376fcc0 thru 0x377df01 (0xe242 bytes) 2024-09-21T22:51:36.802 INFO:teuthology.orchestra.run.smithi094.stdout:5515 write 0x49e006a thru 0x49e43d8 (0x436f bytes) 2024-09-21T22:51:36.804 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.804 INFO:teuthology.orchestra.run.smithi094.stdout:5521 write 0xb9dc95a thru 0xb9e69e2 (0xa089 bytes) 2024-09-21T22:51:36.816 INFO:teuthology.orchestra.run.smithi094.stdout:5524 writesame 0xdcf8866 thru 0xdcfede2 data_size 0x1(0x657d bytes) 2024-09-21T22:51:36.818 INFO:teuthology.orchestra.run.smithi094.stdout:5525 read 0xb0593d7 thru 0xb063846 (0xa470 bytes) 2024-09-21T22:51:36.819 INFO:teuthology.orchestra.run.smithi094.stdout:5526 read 0xd3a8491 thru 0xd3b3fb4 (0xbb24 bytes) 2024-09-21T22:51:36.819 INFO:teuthology.orchestra.run.smithi094.stdout:5527 write 0x1d85c16 thru 0x1d92f8d (0xd378 bytes) 2024-09-21T22:51:36.821 INFO:teuthology.orchestra.run.smithi094.stdout:5528 write 0x2b890fb thru 0x2b9353b (0xa441 bytes) 2024-09-21T22:51:36.824 INFO:teuthology.orchestra.run.smithi094.stdout:5529 writesame 0xac6d372 thru 0xac7533a data_size 0x1(0x7fc9 bytes) 2024-09-21T22:51:36.844 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:36.840+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:36.854 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:36.855 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:36.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:36.889 INFO:teuthology.orchestra.run.smithi094.stdout:5532 trunc from 0xdcfede3 to 0xf6a3ca 2024-09-21T22:51:36.913 INFO:teuthology.orchestra.run.smithi094.stdout:5533 read 0x234c45 thru 0x2431ef (0xe5ab bytes) 2024-09-21T22:51:36.939 INFO:teuthology.orchestra.run.smithi094.stdout:5534 writesame 0x968c99a thru 0x969bb01 data_size 0x96(0xf168 bytes) 2024-09-21T22:51:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:5537 read 0x3e4a400 thru 0x3e54c30 (0xa831 bytes) 2024-09-21T22:51:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:5540 read 0x179bab4 thru 0x17a8353 (0xc8a0 bytes) 2024-09-21T22:51:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:5542 writesame 0x134c10b thru 0x1357b1d data_size 0x7(0xba13 bytes) 2024-09-21T22:51:36.952 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:36.952 INFO:teuthology.orchestra.run.smithi094.stdout:5544 read 0x6f7cbe4 thru 0x6f8ad2f (0xe14c bytes) 2024-09-21T22:51:36.952 INFO:teuthology.orchestra.run.smithi094.stdout:5545 read 0x29da602 thru 0x29dc81c (0x221b bytes) 2024-09-21T22:51:36.952 INFO:teuthology.orchestra.run.smithi094.stdout:5546 read 0x3f28ae4 thru 0x3f2ecb0 (0x61cd bytes) 2024-09-21T22:51:36.961 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:36.959+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:36.967 INFO:teuthology.orchestra.run.smithi094.stdout:5547 write 0xc30d02b thru 0xc311397 (0x436d bytes) 2024-09-21T22:51:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:5548 read 0x318a35e thru 0x3193626 (0x92c9 bytes) 2024-09-21T22:51:36.998 INFO:teuthology.orchestra.run.smithi094.stdout:5549 trunc from 0xc311398 to 0xd20aed 2024-09-21T22:51:37.023 INFO:teuthology.orchestra.run.smithi094.stdout:5550 read 0x937c27 thru 0x947084 (0xf45e bytes) 2024-09-21T22:51:37.024 INFO:teuthology.orchestra.run.smithi094.stdout:5552 punch from 0x22cc1 to 0x326a5, (0xf9e4 bytes) 2024-09-21T22:51:37.043 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:37.043 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:37.045 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:37.043+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:37.058 INFO:teuthology.orchestra.run.smithi094.stdout:5553 write 0x8c87d4c thru 0x8c8ade4 (0x3099 bytes) 2024-09-21T22:51:37.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:37.065 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.065 INFO:teuthology.orchestra.run.smithi094.stdout:5555 read 0x3e92394 thru 0x3e95133 (0x2da0 bytes) 2024-09-21T22:51:37.065 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.065 INFO:teuthology.orchestra.run.smithi094.stdout:5559 write 0x59acaf3 thru 0x59b7611 (0xab1f bytes) 2024-09-21T22:51:37.098 INFO:teuthology.orchestra.run.smithi094.stdout:5560 writesame 0xe79305e thru 0xe79a6fd data_size 0x68(0x76a0 bytes) 2024-09-21T22:51:37.101 INFO:teuthology.orchestra.run.smithi094.stdout:5561 punch from 0xb8be90f to 0xb8c7a5f, (0x9150 bytes) 2024-09-21T22:51:37.101 INFO:teuthology.orchestra.run.smithi094.stdout:5562 write 0xcfbfaf0 thru 0xcfc6dca (0x72db bytes) 2024-09-21T22:51:37.130 INFO:teuthology.orchestra.run.smithi094.stdout:5564 trunc from 0xe79a6fe to 0x4e322de 2024-09-21T22:51:37.146 INFO:teuthology.orchestra.run.smithi094.stdout:5566 punch from 0x14359a4 to 0x143a20d, (0x4869 bytes) 2024-09-21T22:51:37.146 INFO:teuthology.orchestra.run.smithi094.stdout:5567 punch from 0x2c31219 to 0x2c34240, (0x3027 bytes) 2024-09-21T22:51:37.146 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:37.140+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:5570 write 0x80734d6 thru 0x807e470 (0xaf9b bytes) 2024-09-21T22:51:37.166 INFO:teuthology.orchestra.run.smithi094.stdout:5571 read 0x5f1b120 thru 0x5f1baf4 (0x9d5 bytes) 2024-09-21T22:51:37.167 INFO:teuthology.orchestra.run.smithi094.stdout:5572 write 0x24e12e4 thru 0x24ea1b4 (0x8ed1 bytes) 2024-09-21T22:51:37.174 INFO:teuthology.orchestra.run.smithi094.stdout:5573 punch from 0x74f3188 to 0x74fe436, (0xb2ae bytes) 2024-09-21T22:51:37.194 INFO:teuthology.orchestra.run.smithi094.stdout:5575 write 0xb09231b thru 0xb096342 (0x4028 bytes) 2024-09-21T22:51:37.199 INFO:teuthology.orchestra.run.smithi094.stdout:5578 punch from 0x47c69ec to 0x47c90c4, (0x26d8 bytes) 2024-09-21T22:51:37.199 INFO:teuthology.orchestra.run.smithi094.stdout:5579 read 0x1bef18d thru 0x1bf9e6b (0xacdf bytes) 2024-09-21T22:51:37.199 INFO:teuthology.orchestra.run.smithi094.stdout:5581 read 0x91696d5 thru 0x91772a9 (0xdbd5 bytes) 2024-09-21T22:51:37.199 INFO:teuthology.orchestra.run.smithi094.stdout:5582 read 0x466bbb thru 0x4680de (0x1524 bytes) 2024-09-21T22:51:37.200 INFO:teuthology.orchestra.run.smithi094.stdout:5583 writesame 0x256bf28 thru 0x25738bc data_size 0xf(0x7995 bytes) 2024-09-21T22:51:37.222 INFO:teuthology.orchestra.run.smithi094.stdout:5584 write 0xd738c76 thru 0xd73d866 (0x4bf1 bytes) 2024-09-21T22:51:37.225 INFO:teuthology.orchestra.run.smithi094.stdout:5585 read 0x17cd019 thru 0x17d1066 (0x404e bytes) 2024-09-21T22:51:37.234 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.210861757000004, 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.206624187999999, 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.206429290000003, 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484754661, 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:51:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484712781000001, 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484677195, 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:51:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:51:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484645013, 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:51:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:51:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484612987, 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484542975, 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:51:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484479095999999, 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:51:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:51:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484425852999999, 2024-09-21T22:51:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484374544, 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484323719000001, 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:51:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.48427656, 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:51:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:51:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484241023999999, 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:51:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484192795, 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:51:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484124653, 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:51:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:51:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484063516999999, 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:51:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:51:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:51:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.484005349, 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.4839492, 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:51:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:37.327 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5587 write 0xe14bbe7 thru 0xe150fa0 (0x53ba bytes) 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5589 write 0x7f24cb3 thru 0x7f2eda7 (0xa0f5 bytes) 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5590 write 0x7280be8 thru 0x728c7e2 (0xbbfb bytes) 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5592 write 0x162d3f7 thru 0x162f446 (0x2050 bytes) 2024-09-21T22:51:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5594 trunc from 0xe150fa1 to 0x4d6c4b 2024-09-21T22:51:37.330 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:37.324+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:37.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:37.371 INFO:teuthology.orchestra.run.smithi094.stdout:5595 write 0xb454547 thru 0xb456b84 (0x263e bytes) 2024-09-21T22:51:37.377 INFO:teuthology.orchestra.run.smithi094.stdout:5596 writesame 0x20ae6cb thru 0x20b041b data_size 0x13(0x1d51 bytes) 2024-09-21T22:51:37.384 INFO:teuthology.orchestra.run.smithi094.stdout:5598 trunc from 0xb456b85 to 0xce0029e 2024-09-21T22:51:37.397 INFO:teuthology.orchestra.run.smithi094.stdout:5602 write 0x7b8dec7 thru 0x7b9de3b (0xff75 bytes) 2024-09-21T22:51:37.404 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.404 INFO:teuthology.orchestra.run.smithi094.stdout:5604 read 0x224e7be thru 0x2257c5d (0x94a0 bytes) 2024-09-21T22:51:37.420 INFO:teuthology.orchestra.run.smithi094.stdout:5605 write 0xedae23b thru 0xedb508a (0x6e50 bytes) 2024-09-21T22:51:37.426 INFO:teuthology.orchestra.run.smithi094.stdout:5607 writesame 0x2a52c13 thru 0x2a5c3b4 data_size 0xd(0x97a2 bytes) 2024-09-21T22:51:37.429 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:37.423+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:37.433 INFO:teuthology.orchestra.run.smithi094.stdout:5608 read 0xcc65151 thru 0xcc6b80f (0x66bf bytes) 2024-09-21T22:51:37.433 INFO:teuthology.orchestra.run.smithi094.stdout:5609 writesame 0x8417985 thru 0x84198af data_size 0x1(0x1f2b bytes) 2024-09-21T22:51:37.445 INFO:teuthology.orchestra.run.smithi094.stdout:5610 write 0xdfa869a thru 0xdfac635 (0x3f9c bytes) 2024-09-21T22:51:37.449 INFO:teuthology.orchestra.run.smithi094.stdout:5611 read 0x24fe602 thru 0x250536a (0x6d69 bytes) 2024-09-21T22:51:37.449 INFO:teuthology.orchestra.run.smithi094.stdout:5612 write 0x6be6e0c thru 0x6bea5f4 (0x37e9 bytes) 2024-09-21T22:51:37.466 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:37.466 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:37.466 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:37.466 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:37.471 INFO:teuthology.orchestra.run.smithi094.stdout:5616 trunc from 0xedb508b to 0x6aff623 2024-09-21T22:51:37.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:37.489 INFO:teuthology.orchestra.run.smithi094.stdout:5617 read 0x4d752f7 thru 0x4d79d7c (0x4a86 bytes) 2024-09-21T22:51:37.489 INFO:teuthology.orchestra.run.smithi094.stdout:5618 write 0x5e3c6c8 thru 0x5e47106 (0xaa3f bytes) 2024-09-21T22:51:37.492 INFO:teuthology.orchestra.run.smithi094.stdout:5619 write 0x12ca675 thru 0x12cd80f (0x319b bytes) 2024-09-21T22:51:37.495 INFO:teuthology.orchestra.run.smithi094.stdout:5622 read 0x34dfd4d thru 0x34ea1eb (0xa49f bytes) 2024-09-21T22:51:37.497 INFO:teuthology.orchestra.run.smithi094.stdout:5623 writesame 0x238eb6a thru 0x23941c1 data_size 0x1(0x5658 bytes) 2024-09-21T22:51:37.513 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.513 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.529 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:37.523+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:5627 write 0xab9e10e thru 0xab9f596 (0x1489 bytes) 2024-09-21T22:51:37.537 INFO:teuthology.orchestra.run.smithi094.stdout:5629 read 0x42ce891 thru 0x42d2679 (0x3de9 bytes) 2024-09-21T22:51:37.551 INFO:teuthology.orchestra.run.smithi094.stdout:5630 trunc from 0xab9f597 to 0x115a8c0 2024-09-21T22:51:37.616 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:37.616 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:37.616 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:37.616 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:37.616 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:37.627 INFO:teuthology.orchestra.run.smithi094.stdout:5632 writesame 0xbf07d62 thru 0xbf162bb data_size 0x1f(0xe55a bytes) 2024-09-21T22:51:37.627 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.627 INFO:teuthology.orchestra.run.smithi094.stdout:5635 punch from 0x1ad0c44 to 0x1add6c4, (0xca80 bytes) 2024-09-21T22:51:37.630 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:37.627+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:37.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:37.645 INFO:teuthology.orchestra.run.smithi094.stdout:5636 writesame 0xd1c7ae8 thru 0xd1d75b8 data_size 0x11(0xfad1 bytes) 2024-09-21T22:51:37.645 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.645 INFO:teuthology.orchestra.run.smithi094.stdout:5638 write 0xa4ee012 thru 0xa4f76cf (0x96be bytes) 2024-09-21T22:51:37.651 INFO:teuthology.orchestra.run.smithi094.stdout:5639 writesame 0xbd4b3a0 thru 0xbd4d0fd data_size 0x15(0x1d5e bytes) 2024-09-21T22:51:37.657 INFO:teuthology.orchestra.run.smithi094.stdout:5641 punch from 0xc800bda to 0xc80465d, (0x3a83 bytes) 2024-09-21T22:51:37.657 INFO:teuthology.orchestra.run.smithi094.stdout:5642 write 0x3c819a5 thru 0x3c89553 (0x7baf bytes) 2024-09-21T22:51:37.664 INFO:teuthology.orchestra.run.smithi094.stdout:5643 writesame 0x8e03402 thru 0x8e0e3a2 data_size 0x7(0xafa1 bytes) 2024-09-21T22:51:37.672 INFO:teuthology.orchestra.run.smithi094.stdout:5644 writesame 0x2b3d1ac thru 0x2b3df67 data_size 0x6(0xdbc bytes) 2024-09-21T22:51:37.673 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.674 INFO:teuthology.orchestra.run.smithi094.stdout:5647 writesame 0x34b25bd thru 0x34bbbdb data_size 0x1(0x961f bytes) 2024-09-21T22:51:37.716 INFO:teuthology.orchestra.run.smithi094.stdout:5648 write 0x4261b55 thru 0x426c105 (0xa5b1 bytes) 2024-09-21T22:51:37.718 INFO:teuthology.orchestra.run.smithi094.stdout:5649 read 0x1496e70 thru 0x14a5e5d (0xefee bytes) 2024-09-21T22:51:37.720 INFO:teuthology.orchestra.run.smithi094.stdout:5650 trunc from 0xd1d75b9 to 0xcb391f2 2024-09-21T22:51:37.731 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:37.724+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:37.735 INFO:teuthology.orchestra.run.smithi094.stdout:5652 read 0x2f567a9 thru 0x2f5b8b1 (0x5109 bytes) 2024-09-21T22:51:37.736 INFO:teuthology.orchestra.run.smithi094.stdout:5655 writesame 0x33ad812 thru 0x33b7c81 data_size 0x18(0xa470 bytes) 2024-09-21T22:51:37.756 INFO:teuthology.orchestra.run.smithi094.stdout:5656 write 0xe0343b7 thru 0xe040d6b (0xc9b5 bytes) 2024-09-21T22:51:37.765 INFO:teuthology.orchestra.run.smithi094.stdout:5657 punch from 0x2839508 to 0x283fea6, (0x699e bytes) 2024-09-21T22:51:37.765 INFO:teuthology.orchestra.run.smithi094.stdout:5658 write 0x601d138 thru 0x602964b (0xc514 bytes) 2024-09-21T22:51:37.773 INFO:teuthology.orchestra.run.smithi094.stdout:5659 write 0x5dd6ca1 thru 0x5de2ece (0xc22e bytes) 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:37.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.486684533999998, 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:37.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:51:37.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.650424921000001, 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:37.782 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:51:37.787 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536747907000001, 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:51:37.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:51:37.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536599344999999, 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:51:37.792 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:51:37.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:51:37.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536528965, 2024-09-21T22:51:37.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:51:37.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536498752, 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:51:37.800 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.804 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536462492, 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:51:37.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536430799, 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:51:37.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:51:37.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536398566000001, 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536368337000001, 2024-09-21T22:51:37.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.536331743, 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:37.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:51:37.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8289552889999996, 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.824 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8288448489999993, 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:51:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8287679929999996, 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:51:37.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8287001610000004, 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8286298910000003, 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:51:37.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:51:37.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8285452380000002, 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:51:37.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:51:37.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8284848619999998, 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:37.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:37.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8284263190000001, 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:51:37.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8283403759999999, 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:51:37.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:51:37.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.863 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:37.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:37.864 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:37.865 INFO:teuthology.orchestra.run.smithi094.stdout:5660 write 0x10b3376 thru 0x10bfd2b (0xc9b6 bytes) 2024-09-21T22:51:37.865 INFO:teuthology.orchestra.run.smithi094.stdout:5661 read 0x300c47d thru 0x301abe2 (0xe766 bytes) 2024-09-21T22:51:37.865 INFO:teuthology.orchestra.run.smithi094.stdout:5662 write 0x4c52b7a thru 0x4c55b58 (0x2fdf bytes) 2024-09-21T22:51:37.865 INFO:teuthology.orchestra.run.smithi094.stdout:5663 punch from 0x9e81492 to 0x9e89ba2, (0x8710 bytes) 2024-09-21T22:51:37.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5664 write 0xe1efd95 thru 0xe1f21f2 (0x245e bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5665 write 0x5377527 thru 0x5382bbd (0xb697 bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5666 write 0x285162c thru 0x285ab19 (0x94ee bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5667 writesame 0xdd09386 thru 0xdd0dbd9 data_size 0x6(0x4854 bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5668 write 0xb578604 thru 0xb57f6cc (0x70c9 bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5670 read 0x5ab1a89 thru 0x5ab505c (0x35d4 bytes) 2024-09-21T22:51:37.867 INFO:teuthology.orchestra.run.smithi094.stdout:5673 punch from 0x3eb8ded to 0x3ebea7a, (0x5c8d bytes) 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5674 trunc from 0xe1f21f3 to 0xc92577b 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5675 read 0x3cde146 thru 0x3ced207 (0xf0c2 bytes) 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5677 write 0x7718b6a thru 0x771b375 (0x280c bytes) 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5678 read 0xc5dbfd8 thru 0xc5dda5b (0x1a84 bytes) 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5679 punch from 0x12d821d to 0x12dc8eb, (0x46ce bytes) 2024-09-21T22:51:37.868 INFO:teuthology.orchestra.run.smithi094.stdout:5680 write 0xcf40724 thru 0xcf4cd40 (0xc61d bytes) 2024-09-21T22:51:37.868 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:37.860+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:37.884 INFO:teuthology.orchestra.run.smithi094.stdout:5681 write 0xe31337f thru 0xe31713a (0x3dbc bytes) 2024-09-21T22:51:37.887 INFO:teuthology.orchestra.run.smithi094.stdout:5683 writesame 0xe177a2a thru 0xe17e3f4 data_size 0x35(0x69cb bytes) 2024-09-21T22:51:37.891 INFO:teuthology.orchestra.run.smithi094.stdout:5684 read 0x1dd4904 thru 0x1dd5322 (0xa1f bytes) 2024-09-21T22:51:37.892 INFO:teuthology.orchestra.run.smithi094.stdout:5685 write 0x980b533 thru 0x98106bc (0x518a bytes) 2024-09-21T22:51:37.896 INFO:teuthology.orchestra.run.smithi094.stdout:5687 write 0x11ca7d7 thru 0x11d8926 (0xe150 bytes) 2024-09-21T22:51:37.900 INFO:teuthology.orchestra.run.smithi094.stdout:5688 read 0x269b660 thru 0x269ecc4 (0x3665 bytes) 2024-09-21T22:51:37.901 INFO:teuthology.orchestra.run.smithi094.stdout:5689 read 0xdde301c thru 0xdde8b0b (0x5af0 bytes) 2024-09-21T22:51:37.901 INFO:teuthology.orchestra.run.smithi094.stdout:5690 read 0x8d69a10 thru 0x8d6e416 (0x4a07 bytes) 2024-09-21T22:51:37.902 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:37.902 INFO:teuthology.orchestra.run.smithi094.stdout:5692 write 0xdbef1b1 thru 0xdbfa798 (0xb5e8 bytes) 2024-09-21T22:51:37.904 INFO:teuthology.orchestra.run.smithi094.stdout:5693 write 0x9ccda2a thru 0x9cd9cf4 (0xc2cb bytes) 2024-09-21T22:51:37.908 INFO:teuthology.orchestra.run.smithi094.stdout:5697 read 0x739ab60 thru 0x739c832 (0x1cd3 bytes) 2024-09-21T22:51:37.909 INFO:teuthology.orchestra.run.smithi094.stdout:5698 read 0x83339b7 thru 0x833cc6b (0x92b5 bytes) 2024-09-21T22:51:37.910 INFO:teuthology.orchestra.run.smithi094.stdout:5699 write 0x1563673 thru 0x156c4da (0x8e68 bytes) 2024-09-21T22:51:37.914 INFO:teuthology.orchestra.run.smithi094.stdout:5700 writesame 0xca20986 thru 0xca2b342 data_size 0x1(0xa9bd bytes) 2024-09-21T22:51:37.957 INFO:teuthology.orchestra.run.smithi094.stdout:5703 writesame 0x66f3d0e thru 0x66fe9e9 data_size 0x5c(0xacdc bytes) 2024-09-21T22:51:37.962 INFO:teuthology.orchestra.run.smithi094.stdout:5704 write 0xdf86f42 thru 0xdf93785 (0xc844 bytes) 2024-09-21T22:51:37.966 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:37.960+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:37.966 INFO:teuthology.orchestra.run.smithi094.stdout:5705 writesame 0x7e9ea55 thru 0x7eae35f data_size 0x29(0xf90b bytes) 2024-09-21T22:51:37.971 INFO:teuthology.orchestra.run.smithi094.stdout:5706 write 0xb7184a thru 0xb75f2d (0x46e4 bytes) 2024-09-21T22:51:37.975 INFO:teuthology.orchestra.run.smithi094.stdout:5707 write 0x95c99c0 thru 0x95d5407 (0xba48 bytes) 2024-09-21T22:51:38.011 INFO:teuthology.orchestra.run.smithi094.stdout:5708 trunc from 0xe31713b to 0x339067 2024-09-21T22:51:38.017 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.017 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:38.017 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:38.018 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:38.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:38.063 INFO:teuthology.orchestra.run.smithi094.stdout:5709 punch from 0x16ef27 to 0x178c21, (0x9cfa bytes) 2024-09-21T22:51:38.063 INFO:teuthology.orchestra.run.smithi094.stdout:5710 trunc from 0x339067 to 0x2f31de 2024-09-21T22:51:38.067 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:38.063+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:38.077 INFO:teuthology.orchestra.run.smithi094.stdout:5711 punch from 0x204a93 to 0x2074e0, (0x2a4d bytes) 2024-09-21T22:51:38.099 INFO:teuthology.orchestra.run.smithi094.stdout:5714 writesame 0x5b54cfa thru 0x5b5c5ab data_size 0x7(0x78b2 bytes) 2024-09-21T22:51:38.099 INFO:teuthology.orchestra.run.smithi094.stdout:5715 read 0x5ce1a6 thru 0x5d89b7 (0xa812 bytes) 2024-09-21T22:51:38.099 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.124 INFO:teuthology.orchestra.run.smithi094.stdout:5719 write 0xd0636f7 thru 0xd070c73 (0xd57d bytes) 2024-09-21T22:51:38.129 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.129 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.130 INFO:teuthology.orchestra.run.smithi094.stdout:5725 read 0x504124a thru 0x5041d5f (0xb16 bytes) 2024-09-21T22:51:38.130 INFO:teuthology.orchestra.run.smithi094.stdout:5726 read 0x2a1209c thru 0x2a200ea (0xe04f bytes) 2024-09-21T22:51:38.130 INFO:teuthology.orchestra.run.smithi094.stdout:5727 read 0xc2c0a24 thru 0xc2c3e9f (0x347c bytes) 2024-09-21T22:51:38.130 INFO:teuthology.orchestra.run.smithi094.stdout:5728 read 0x590618f thru 0x5912db8 (0xcc2a bytes) 2024-09-21T22:51:38.130 INFO:teuthology.orchestra.run.smithi094.stdout:5729 writesame 0xccafc8e thru 0xccbf6f5 data_size 0xc(0xfa68 bytes) 2024-09-21T22:51:38.153 INFO:teuthology.orchestra.run.smithi094.stdout:5730 writesame 0xdf244e9 thru 0xdf29242 data_size 0x1(0x4d5a bytes) 2024-09-21T22:51:38.155 INFO:teuthology.orchestra.run.smithi094.stdout:5732 read 0xd4a4e35 thru 0xd4b30b4 (0xe280 bytes) 2024-09-21T22:51:38.155 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.155 INFO:teuthology.orchestra.run.smithi094.stdout:5734 read 0x459caf4 thru 0x45a5d4f (0x925c bytes) 2024-09-21T22:51:38.156 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.156 INFO:teuthology.orchestra.run.smithi094.stdout:5736 read 0x7c2006c thru 0x7c279d0 (0x7965 bytes) 2024-09-21T22:51:38.156 INFO:teuthology.orchestra.run.smithi094.stdout:5737 writesame 0x56359c8 thru 0x5644ad1 data_size 0x1(0xf10a bytes) 2024-09-21T22:51:38.167 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:38.163+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:38.191 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.191 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:38.191 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:38.191 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:38.191 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:38.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:38.247 INFO:teuthology.orchestra.run.smithi094.stdout:5738 writesame 0xed53977 thru 0xed53a52 data_size 0xa(0xdc bytes) 2024-09-21T22:51:38.247 INFO:teuthology.orchestra.run.smithi094.stdout:5741 read 0x435974d thru 0x4366e0f (0xd6c3 bytes) 2024-09-21T22:51:38.248 INFO:teuthology.orchestra.run.smithi094.stdout:5746 write 0x8ef751e thru 0x8f02e9e (0xb981 bytes) 2024-09-21T22:51:38.250 INFO:teuthology.orchestra.run.smithi094.stdout:5747 write 0xd835b90 thru 0xd843a29 (0xde9a bytes) 2024-09-21T22:51:38.253 INFO:teuthology.orchestra.run.smithi094.stdout:5748 punch from 0xeaa4ed1 to 0xeaacd9f, (0x7ece bytes) 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:5749 punch from 0xacf4dcc to 0xacfd0ce, (0x8302 bytes) 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:5751 punch from 0x47dd164 to 0x47e7881, (0xa71d bytes) 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:5753 read 0x67b5ca9 thru 0x67c4b3d (0xee95 bytes) 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:5754 read 0xb2759b thru 0xb29e18 (0x287e bytes) 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.254 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.255 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.255 INFO:teuthology.orchestra.run.smithi094.stdout:5761 writesame 0xe0e1b85 thru 0xe0ed98f data_size 0x1(0xbe0b bytes) 2024-09-21T22:51:38.268 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:38.263+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:38.312 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.312 INFO:teuthology.orchestra.run.smithi094.stdout:5763 write 0x70ed530 thru 0x70f42b5 (0x6d86 bytes) 2024-09-21T22:51:38.317 INFO:teuthology.orchestra.run.smithi094.stdout:5764 write 0x78d473f thru 0x78d4c79 (0x53b bytes) 2024-09-21T22:51:38.323 INFO:teuthology.orchestra.run.smithi094.stdout:5765 read 0xcfe6d73 thru 0xcfee6be (0x794c bytes) 2024-09-21T22:51:38.329 INFO:teuthology.orchestra.run.smithi094.stdout:5767 trunc from 0xed53a53 to 0xb4305cf 2024-09-21T22:51:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:5768 read 0x2d60e63 thru 0x2d630ea (0x2288 bytes) 2024-09-21T22:51:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:5769 read 0x24c5983 thru 0x24c953d (0x3bbb bytes) 2024-09-21T22:51:38.364 INFO:teuthology.orchestra.run.smithi094.stdout:5771 writesame 0xebb6f4a thru 0xebc20bb data_size 0x43(0xb172 bytes) 2024-09-21T22:51:38.364 INFO:teuthology.orchestra.run.smithi094.stdout:5772 read 0xdadf767 thru 0xdae29e2 (0x327c bytes) 2024-09-21T22:51:38.364 INFO:teuthology.orchestra.run.smithi094.stdout:5773 write 0x21fc5d8 thru 0x2202237 (0x5c60 bytes) 2024-09-21T22:51:38.368 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:38.364+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 53.281408841999998, 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:38.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:38.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.881208977, 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:38.375 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:38.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879694717, 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.380 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:38.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:38.384 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879641415000002, 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:38.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:38.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:38.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879586367000002, 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:38.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:38.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.393 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879517182999997, 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:38.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:38.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879454615999997, 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.399 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:38.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:38.403 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.879405222000003, 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:38.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:38.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.878901439000003, 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.408 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:38.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:38.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.87883386, 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:38.412 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.878284616999998, 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:38.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:38.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.878207136, 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.421 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:38.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:38.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:38.425 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.878043005000002, 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:38.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.877982025999998, 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.877938413999999, 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:38.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:38.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.877893931999999, 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:38.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:38.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.877848763999999, 2024-09-21T22:51:38.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:38.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:38.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.877806466999999, 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.447 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:38.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:38.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:38.451 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.392373878000001, 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:38.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:38.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.040879404, 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:51:38.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.462 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:38.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:51:38.463 INFO:teuthology.orchestra.run.smithi094.stdout:5774 trunc from 0xebc20bc to 0x9cb03b3 2024-09-21T22:51:38.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:38.464 INFO:teuthology.orchestra.run.smithi094.stdout:5775 trunc from 0x9cb03b3 to 0x369622 2024-09-21T22:51:38.469 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:38.467+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:38.479 INFO:teuthology.orchestra.run.smithi094.stdout:5776 writesame 0xa69cfe9 thru 0xa6a8df8 data_size 0x8(0xbe10 bytes) 2024-09-21T22:51:38.479 INFO:teuthology.orchestra.run.smithi094.stdout:5777 read 0x6cae6a5 thru 0x6cb3d47 (0x56a3 bytes) 2024-09-21T22:51:38.479 INFO:teuthology.orchestra.run.smithi094.stdout:5778 write 0xd4b0c1 thru 0xd50fcc (0x5f0c bytes) 2024-09-21T22:51:38.487 INFO:teuthology.orchestra.run.smithi094.stdout:5779 read 0x3e1a009 thru 0x3e22d8e (0x8d86 bytes) 2024-09-21T22:51:38.487 INFO:teuthology.orchestra.run.smithi094.stdout:5780 write 0x84c6f15 thru 0x84cdbbf (0x6cab bytes) 2024-09-21T22:51:38.493 INFO:teuthology.orchestra.run.smithi094.stdout:5782 write 0x7b4f77e thru 0x7b5e7b9 (0xf03c bytes) 2024-09-21T22:51:38.521 INFO:teuthology.orchestra.run.smithi094.stdout:5785 write 0xde5753f thru 0xde5ea70 (0x7532 bytes) 2024-09-21T22:51:38.523 INFO:teuthology.orchestra.run.smithi094.stdout:5786 writesame 0x514e5d9 thru 0x515bc09 data_size 0x1(0xd631 bytes) 2024-09-21T22:51:38.570 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:38.564+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:38.594 INFO:teuthology.orchestra.run.smithi094.stdout:5787 read 0x6de16b0 thru 0x6defea0 (0xe7f1 bytes) 2024-09-21T22:51:38.594 INFO:teuthology.orchestra.run.smithi094.stdout:5788 read 0x2617a45 thru 0x261c621 (0x4bdd bytes) 2024-09-21T22:51:38.594 INFO:teuthology.orchestra.run.smithi094.stdout:5789 read 0x820fb82 thru 0x821d983 (0xde02 bytes) 2024-09-21T22:51:38.594 INFO:teuthology.orchestra.run.smithi094.stdout:5790 write 0x55af1b thru 0x56a739 (0xf81f bytes) 2024-09-21T22:51:38.598 INFO:teuthology.orchestra.run.smithi094.stdout:5791 read 0x70d88ef thru 0x70d98b0 (0xfc2 bytes) 2024-09-21T22:51:38.598 INFO:teuthology.orchestra.run.smithi094.stdout:5792 punch from 0xa978bb4 to 0xa981f1d, (0x9369 bytes) 2024-09-21T22:51:38.598 INFO:teuthology.orchestra.run.smithi094.stdout:5793 writesame 0x7ab28a8 thru 0x7abfd89 data_size 0x6(0xd4e2 bytes) 2024-09-21T22:51:38.604 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:38.605 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:38.611 INFO:teuthology.orchestra.run.smithi094.stdout:5796 read 0x9c97ad9 thru 0x9c9cddd (0x5305 bytes) 2024-09-21T22:51:38.612 INFO:teuthology.orchestra.run.smithi094.stdout:5797 writesame 0xb4358fa thru 0xb43c9bc data_size 0x1(0x70c3 bytes) 2024-09-21T22:51:38.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:38.633 INFO:teuthology.orchestra.run.smithi094.stdout:5798 writesame 0x6688d8d thru 0x66900ca data_size 0x63(0x733e bytes) 2024-09-21T22:51:38.642 INFO:teuthology.orchestra.run.smithi094.stdout:5801 writesame 0xd3d9447 thru 0xd3e1c6b data_size 0xd(0x8825 bytes) 2024-09-21T22:51:38.648 INFO:teuthology.orchestra.run.smithi094.stdout:5802 write 0xb3c9a9d thru 0xb3d778c (0xdcf0 bytes) 2024-09-21T22:51:38.651 INFO:teuthology.orchestra.run.smithi094.stdout:5803 write 0x4f6dfa8 thru 0x4f753ab (0x7404 bytes) 2024-09-21T22:51:38.653 INFO:teuthology.orchestra.run.smithi094.stdout:5804 punch from 0xd7df09e to 0xd7ebc23, (0xcb85 bytes) 2024-09-21T22:51:38.653 INFO:teuthology.orchestra.run.smithi094.stdout:5805 punch from 0x47e7d4f to 0x47f6d85, (0xf036 bytes) 2024-09-21T22:51:38.653 INFO:teuthology.orchestra.run.smithi094.stdout:5806 writesame 0x1482cbc thru 0x1483e58 data_size 0x9(0x119d bytes) 2024-09-21T22:51:38.672 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:38.667+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:38.675 INFO:teuthology.orchestra.run.smithi094.stdout:5807 trunc from 0xde5ea71 to 0x2640535 2024-09-21T22:51:38.724 INFO:teuthology.orchestra.run.smithi094.stdout:5810 write 0x7d3bd95 thru 0x7d47ca3 (0xbf0f bytes) 2024-09-21T22:51:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:5811 punch from 0x73ac932 to 0x73b3ace, (0x719c bytes) 2024-09-21T22:51:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:5813 writesame 0x3ace55c thru 0x3ad6345 data_size 0x2(0x7dea bytes) 2024-09-21T22:51:38.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:38.751 INFO:teuthology.orchestra.run.smithi094.stdout:5814 writesame 0x4a62c15 thru 0x4a6a610 data_size 0x4a(0x79fc bytes) 2024-09-21T22:51:38.754 INFO:teuthology.orchestra.run.smithi094.stdout:5816 read 0x6710560 thru 0x67183de (0x7e7f bytes) 2024-09-21T22:51:38.755 INFO:teuthology.orchestra.run.smithi094.stdout:5818 punch from 0x37d14d7 to 0x37d39f8, (0x2521 bytes) 2024-09-21T22:51:38.756 INFO:teuthology.orchestra.run.smithi094.stdout:5819 read 0x4d5ff5d thru 0x4d6ddb1 (0xde55 bytes) 2024-09-21T22:51:38.756 INFO:teuthology.orchestra.run.smithi094.stdout:5820 read 0x4cbbcae thru 0x4cc9fa4 (0xe2f7 bytes) 2024-09-21T22:51:38.756 INFO:teuthology.orchestra.run.smithi094.stdout:5821 read 0x63f57de thru 0x640509c (0xf8bf bytes) 2024-09-21T22:51:38.757 INFO:teuthology.orchestra.run.smithi094.stdout:5822 punch from 0x238e3e1 to 0x2393704, (0x5323 bytes) 2024-09-21T22:51:38.758 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.758 INFO:teuthology.orchestra.run.smithi094.stdout:5825 write 0x3070a0f thru 0x30730bb (0x26ad bytes) 2024-09-21T22:51:38.760 INFO:teuthology.orchestra.run.smithi094.stdout:5826 write 0x74f6889 thru 0x74ff11e (0x8896 bytes) 2024-09-21T22:51:38.765 INFO:teuthology.orchestra.run.smithi094.stdout:5828 punch from 0x5e40d4d to 0x5e4a8f1, (0x9ba4 bytes) 2024-09-21T22:51:38.766 INFO:teuthology.orchestra.run.smithi094.stdout:5830 write 0x1a5d155 thru 0x1a5f4c6 (0x2372 bytes) 2024-09-21T22:51:38.769 INFO:teuthology.orchestra.run.smithi094.stdout:5832 write 0x5a8b6e0 thru 0x5a913b2 (0x5cd3 bytes) 2024-09-21T22:51:38.773 INFO:teuthology.orchestra.run.smithi094.stdout:5834 read 0xaa799e thru 0xaa90a5 (0x1708 bytes) 2024-09-21T22:51:38.773 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:38.767+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:38.773 INFO:teuthology.orchestra.run.smithi094.stdout:5835 punch from 0x1e1cd98 to 0x1e20d44, (0x3fac bytes) 2024-09-21T22:51:38.773 INFO:teuthology.orchestra.run.smithi094.stdout:5836 write 0x77fea7c thru 0x780e58e (0xfb13 bytes) 2024-09-21T22:51:38.775 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.775 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:38.775 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:38.775 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:38.775 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:38.776 INFO:teuthology.orchestra.run.smithi094.stdout:5838 write 0x3bc84cc thru 0x3bccbc0 (0x46f5 bytes) 2024-09-21T22:51:38.787 INFO:teuthology.orchestra.run.smithi094.stdout:5839 trunc from 0x7d47ca4 to 0xd26c220 2024-09-21T22:51:38.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:38.799 INFO:teuthology.orchestra.run.smithi094.stdout:5840 write 0x9da7781 thru 0x9dae9d5 (0x7255 bytes) 2024-09-21T22:51:38.803 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:38.803 INFO:teuthology.orchestra.run.smithi094.stdout:5845 read 0x94624b1 thru 0x9464fad (0x2afd bytes) 2024-09-21T22:51:38.816 INFO:teuthology.orchestra.run.smithi094.stdout:5846 write 0xe654baf thru 0xe65d513 (0x8965 bytes) 2024-09-21T22:51:38.840 INFO:teuthology.orchestra.run.smithi094.stdout:5847 trunc from 0xe65d514 to 0xf83225 2024-09-21T22:51:38.874 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:38.868+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:38.905 INFO:teuthology.orchestra.run.smithi094.stdout:5848 writesame 0x56c6c8c thru 0x56c78ee data_size 0x7(0xc63 bytes) 2024-09-21T22:51:38.908 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:38.908 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:38.917 INFO:teuthology.orchestra.run.smithi094.stdout:5849 trunc from 0x56c78ef to 0xc0bc0d0 2024-09-21T22:51:38.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:38.925 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:38.925 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:38.927 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:38.927 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:38.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 80.699061427999993, 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:38.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.882498429999998, 2024-09-21T22:51:38.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.936 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.705533291999998, 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:38.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:38.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:38.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 49.094424635999999, 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:38.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:38.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:38.944 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.914619436999999, 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.836999765999998, 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.949 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.836734495000002, 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.506661698000002, 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.325575747999999, 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:38.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:38.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:38.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.310960381999998, 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:38.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.310577791, 2024-09-21T22:51:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:38.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:38.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.308565326999997, 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:38.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.279840526999998, 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:38.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:38.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 43.277147757000002, 2024-09-21T22:51:38.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:38.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:38.986 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.516688132999999, 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:38.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8297667430000004, 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:51:38.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168816 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'545, mlcod=87'543)", 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.0350239120000002, 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033834189999999999, 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:38.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891267+0000", 2024-09-21T22:51:38.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2500000000000001e-06 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891281+0000", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3723000000000001e-05 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891323+0000", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2926000000000002e-05 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891344+0000", 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0250000000000001e-05 2024-09-21T22:51:38.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891475+0000", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013148000000000001 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891506+0000", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0784999999999997e-05 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894584+0000", 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003077598 2024-09-21T22:51:38.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894648+0000", 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4406999999999995e-05 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168818 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'547, mlcod=87'543)", 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.0330635560000001, 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039584049999999999, 2024-09-21T22:51:38.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893227+0000", 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7460000000000001e-06 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893242+0000", 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5067e-05 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:39.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893273+0000", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0920000000000002e-05 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893292+0000", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9732999999999999e-05 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893858+0000", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000565209 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:39.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893891+0000", 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3822999999999998e-05 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897073+0000", 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031815010000000002 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897183+0000", 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110406 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:39.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168827 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'552, mlcod=87'549)", 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.025592413, 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003413499, 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900698+0000", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.787e-06 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900712+0000", 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4486e-05 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900746+0000", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3330000000000001e-05 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900765+0000", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.906e-05 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900904+0000", 2024-09-21T22:51:39.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139519 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900980+0000", 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5636000000000006e-05 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904045+0000", 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030655510000000001 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:39.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904110+0000", 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4129999999999998e-05 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:169241 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'653, mlcod=87'650)", 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.89825091700000004, 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032275400000000001, 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028047+0000", 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7059999999999999e-06 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028061+0000", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.411e-05 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028094+0000", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2292999999999997e-05 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028111+0000", 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:51:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028274+0000", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162442 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028329+0000", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5112999999999999e-05 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031115+0000", 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002785624 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031265+0000", 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150512 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:39.012 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:39.013 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:39.013 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:39.013 INFO:teuthology.orchestra.run.smithi094.stdout:5853 write 0x13b269b thru 0x13b4022 (0x1988 bytes) 2024-09-21T22:51:39.013 INFO:teuthology.orchestra.run.smithi094.stdout:5855 trunc from 0xc0bc0d0 to 0x750bf52 2024-09-21T22:51:39.013 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:39.014 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:39.014 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:39.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:39.015 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:39.015 INFO:teuthology.orchestra.run.smithi094.stdout:5858 punch from 0x2dd96b4 to 0x2de8a2b, (0xf377 bytes) 2024-09-21T22:51:39.015 INFO:teuthology.orchestra.run.smithi094.stdout:5859 read 0x31d1a10 thru 0x31de008 (0xc5f9 bytes) 2024-09-21T22:51:39.015 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5860 writesame 0xabfabb7 thru 0xabfad79 data_size 0x1(0x1c3 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5861 read 0x3807e1b thru 0x3816ace (0xecb4 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5863 read 0x9cc8700 thru 0x9ccfb9a (0x749b bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5865 punch from 0x62e6277 to 0x62eb91f, (0x56a8 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5867 writesame 0x4c53131 thru 0x4c587eb data_size 0x3(0x56bb bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5868 read 0x4a080de thru 0x4a13285 (0xb1a8 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5869 write 0x8d4683e thru 0x8d56352 (0xfb15 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5870 read 0x4f4eefd thru 0x4f519ea (0x2aee bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:5872 writesame 0x9c1df6c thru 0x9c29a13 data_size 0x84(0xbaa8 bytes) 2024-09-21T22:51:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:5874 trunc from 0xabfad7a to 0xd4de304 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:5875 write 0xb1df18d thru 0xb1df2cb (0x13f bytes) 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:5877 read 0x152ebca thru 0x1533d7e (0x51b5 bytes) 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:5881 punch from 0x4f9440d to 0x4f9f7eb, (0xb3de bytes) 2024-09-21T22:51:39.018 INFO:teuthology.orchestra.run.smithi094.stdout:5882 punch from 0x2af0624 to 0x2af55e2, (0x4fbe bytes) 2024-09-21T22:51:39.019 INFO:teuthology.orchestra.run.smithi094.stdout:5883 writesame 0xa4d5f6f thru 0xa4dc117 data_size 0x1(0x61a9 bytes) 2024-09-21T22:51:39.019 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:39.019 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:39.011+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:39.034 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:config diff 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:config diff get 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:config get 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:config help [] 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:config set ... 2024-09-21T22:51:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:admin_socket: invalid command 2024-09-21T22:51:39.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T22:51:39.037 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-21T22:51:39.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:39.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:5886 write 0x22bbe22 thru 0x22bca4f (0xc2e bytes) 2024-09-21T22:51:39.050 INFO:teuthology.orchestra.run.smithi094.stdout:5887 punch from 0x82ac753 to 0x82b9f80, (0xd82d bytes) 2024-09-21T22:51:39.051 INFO:teuthology.orchestra.run.smithi094.stdout:5888 writesame 0x50be961 thru 0x50bfdab data_size 0x1(0x144b bytes) 2024-09-21T22:51:39.067 INFO:teuthology.orchestra.run.smithi094.stdout:5889 trunc from 0xd4de304 to 0x53bf7c2 2024-09-21T22:51:39.080 INFO:teuthology.orchestra.run.smithi094.stdout:5890 trunc from 0x53bf7c2 to 0x79fae6d 2024-09-21T22:51:39.090 INFO:teuthology.orchestra.run.smithi094.stdout:5892 punch from 0x2f43ce6 to 0x2f44a32, (0xd4c bytes) 2024-09-21T22:51:39.090 INFO:teuthology.orchestra.run.smithi094.stdout:5894 read 0x79b620c thru 0x79b8802 (0x25f7 bytes) 2024-09-21T22:51:39.091 INFO:teuthology.orchestra.run.smithi094.stdout:5895 punch from 0x1e44e0e to 0x1e54317, (0xf509 bytes) 2024-09-21T22:51:39.099 INFO:teuthology.orchestra.run.smithi094.stdout:5896 trunc from 0x79fae6d to 0xe63b296 2024-09-21T22:51:39.114 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:39.108+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:39.118 INFO:teuthology.orchestra.run.smithi094.stdout:5897 trunc from 0xe63b296 to 0x63e1065 2024-09-21T22:51:39.144 INFO:teuthology.orchestra.run.smithi094.stdout:5898 write 0x8926479 thru 0x893232b (0xbeb3 bytes) 2024-09-21T22:51:39.146 INFO:teuthology.orchestra.run.smithi094.stdout:5899 punch from 0x6793825 to 0x6796dcc, (0x35a7 bytes) 2024-09-21T22:51:39.146 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:39.154 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:39.155 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:39.162 INFO:teuthology.orchestra.run.smithi094.stdout:5901 writesame 0xd67023d thru 0xd67ab5a data_size 0x1(0xa91e bytes) 2024-09-21T22:51:39.163 INFO:teuthology.orchestra.run.smithi094.stdout:5902 read 0x63a45f thru 0x64a0b3 (0xfc55 bytes) 2024-09-21T22:51:39.163 INFO:teuthology.orchestra.run.smithi094.stdout:5904 read 0x91baaad thru 0x91c8326 (0xd87a bytes) 2024-09-21T22:51:39.163 INFO:teuthology.orchestra.run.smithi094.stdout:5905 read 0xc83342d thru 0xc840f49 (0xdb1d bytes) 2024-09-21T22:51:39.164 INFO:teuthology.orchestra.run.smithi094.stdout:5906 write 0xa6e3dda thru 0xa6ed84a (0x9a71 bytes) 2024-09-21T22:51:39.166 INFO:teuthology.orchestra.run.smithi094.stdout:5907 write 0x912b9da thru 0x9138043 (0xc66a bytes) 2024-09-21T22:51:39.168 INFO:teuthology.orchestra.run.smithi094.stdout:5908 read 0x31ee546 thru 0x31f6457 (0x7f12 bytes) 2024-09-21T22:51:39.168 INFO:teuthology.orchestra.run.smithi094.stdout:5909 clone 33 order 23 su 4194304 sc 4 2024-09-21T22:51:39.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:39.218 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:39.215+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:39.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:39.346 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:39.343+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:39.416 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:39.411+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:39.419 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:51:39.471 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:39.471 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:39.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:39.517 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:39.512+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:39.547 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:39.540+0000 7fb1fe3537c0 -1 osd.3 77 log_to_monitors true 2024-09-21T22:51:39.617 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:39.615+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:39.650 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:39.651 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:39.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:39.692 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T22:51:39.718 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:39.712+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:39.819 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:39.815+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:39.836 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:39.836 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:39.850 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:39.920 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:39.916+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:40.017 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.018 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.021 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:40.016+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:40.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:40.060 INFO:teuthology.orchestra.run.smithi094.stdout:5911 read 0x7898658 thru 0x789b134 (0x2add bytes) 2024-09-21T22:51:40.061 INFO:teuthology.orchestra.run.smithi094.stdout:5914 punch from 0xa10e71a to 0xa1193cd, (0xacb3 bytes) 2024-09-21T22:51:40.067 INFO:teuthology.orchestra.run.smithi094.stdout:5915 punch from 0x2d5d252 to 0x2d5fa07, (0x27b5 bytes) 2024-09-21T22:51:40.078 INFO:teuthology.orchestra.run.smithi094.stdout:5916 write 0xed3d9e4 thru 0xed456a4 (0x7cc1 bytes) 2024-09-21T22:51:40.083 INFO:teuthology.orchestra.run.smithi094.stdout:5917 write 0x5c6b7ce thru 0x5c7a23b (0xea6e bytes) 2024-09-21T22:51:40.088 INFO:teuthology.orchestra.run.smithi094.stdout:5918 write 0x5611c76 thru 0x561a7cd (0x8b58 bytes) 2024-09-21T22:51:40.094 INFO:teuthology.orchestra.run.smithi094.stdout:5919 write 0x74cabda thru 0x74d5b51 (0xaf78 bytes) 2024-09-21T22:51:40.100 INFO:teuthology.orchestra.run.smithi094.stdout:5920 write 0x7fa2e9 thru 0x80a066 (0xfd7e bytes) 2024-09-21T22:51:40.106 INFO:teuthology.orchestra.run.smithi094.stdout:5921 read 0xbaa467 thru 0xbaf307 (0x4ea1 bytes) 2024-09-21T22:51:40.106 INFO:teuthology.orchestra.run.smithi094.stdout:5922 writesame 0x7391c13 thru 0x7397e66 data_size 0x3e(0x6254 bytes) 2024-09-21T22:51:40.112 INFO:teuthology.orchestra.run.smithi094.stdout:5923 read 0x1bcf8f2 thru 0x1bd2516 (0x2c25 bytes) 2024-09-21T22:51:40.112 INFO:teuthology.orchestra.run.smithi094.stdout:5924 punch from 0x45e695e to 0x45ef115, (0x87b7 bytes) 2024-09-21T22:51:40.112 INFO:teuthology.orchestra.run.smithi094.stdout:5926 clone 34 order 24 su 131072 sc 9 2024-09-21T22:51:40.122 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:40.119+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:40.159 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.160 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:40.261 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:40.259+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:40.323 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:40.319+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:40.339 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.339 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:40.438 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:40.435+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:40.464 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:40.460+0000 7fb1f2120640 -1 osd.3 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T22:51:40.516 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.516 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.525 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:40.519+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:40.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:40.579 INFO:teuthology.orchestra.run.smithi094.stdout:flattening image image_client.0-clone33 2024-09-21T22:51:40.626 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:40.623+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:40.701 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.701 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:40.758 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:40.755+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:40.828 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:40.820+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:40.883 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T22:51:40.887 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:40.887 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:40.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:40.950 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:40.947+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:41.023 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:41.023 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:41.029 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:41.023+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:41.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:41.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:41.130 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:41.127+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:41.169 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:41.169 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:41.185 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:41.231 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:41.227+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:41.232 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:41.232 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:41.249 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:41.249 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:41.281 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:41.281 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:41.281 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-21T22:51:41.302 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:41.302 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:41.311 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:41.311 INFO:teuthology.orchestra.run.smithi094.stdout:5928 read 0x46d56f6 thru 0x46d5b1b (0x426 bytes) 2024-09-21T22:51:41.313 INFO:teuthology.orchestra.run.smithi094.stdout:5929 write 0xae96fb3 thru 0xaea4c5d (0xdcab bytes) 2024-09-21T22:51:41.316 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:41.317 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:41.325 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:41.331 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:41.324+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:41.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:41.339 INFO:teuthology.orchestra.run.smithi094.stdout:5930 trunc from 0xed456a5 to 0xca8beb9 2024-09-21T22:51:41.351 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:41.351 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:41.375 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:41.379 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T22:51:41.379 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T22:51:41.384 INFO:teuthology.orchestra.run.smithi094.stdout:5931 punch from 0x43a7793 to 0x43ad588, (0x5df5 bytes) 2024-09-21T22:51:41.385 INFO:teuthology.orchestra.run.smithi094.stdout:5932 clone 35 order 24 su 4194304 sc 3 2024-09-21T22:51:41.432 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:41.428+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:41.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.183152151000002, 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:51:41.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:51:41.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.476 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.346892537999999, 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.233215523999998, 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:51:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.233066961999999, 2024-09-21T22:51:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:51:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:51:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232996581999998, 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:51:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232966369, 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232930109000002, 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:51:41.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:51:41.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232898416000001, 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232866182999999, 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.506 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:51:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:51:41.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.509 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232835953999999, 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:41.513 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.232799360000001, 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:41.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:51:41.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:51:41.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525422905999999, 2024-09-21T22:51:41.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525312466000001, 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525235609999999, 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:51:41.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:51:41.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:51:41.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525167778, 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525097508, 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:51:41.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:51:41.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:51:41.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.525012855, 2024-09-21T22:51:41.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.524952479, 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.524893936, 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.524807993, 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:41.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:41.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:41.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:41.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:41.555 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:41.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:41.557 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:41.552+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:41.657 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:41.652+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:41.720 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:41.721 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:41.721 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:41.721 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:41.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:41.776 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:41.772+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:41.858 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:41.852+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:41.896 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:41.896 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:41.896 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:41.896 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:41.896 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:41.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:41.996 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:41.992+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:42.060 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:42.056+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:42.072 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:42.072 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.981522378000001, 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.581322513000003, 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:42.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:42.081 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579808253000003, 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:42.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:42.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579754950999998, 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.089 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579699902999998, 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579630719000001, 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:42.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579568152, 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.102 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579518757999999, 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:42.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:42.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:42.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.106 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.579014975, 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.110 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578947395999997, 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:42.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578398153000002, 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.115 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:42.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:42.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578320671999997, 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:42.119 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578156540999998, 2024-09-21T22:51:42.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:42.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578095562000001, 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:42.127 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.131 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578051950000003, 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:42.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.578007468000003, 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.136 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:42.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.577962300000003, 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.140 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:42.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.577920003000003, 2024-09-21T22:51:42.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.092487413999997, 2024-09-21T22:51:42.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:42.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:42.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:42.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.153 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.740992940000002, 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:51:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:51:42.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.158 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:42.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:42.161 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:42.159+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:42.261 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:42.256+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:42.324 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:42.324 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:42.324 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:42.324 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:42.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:42.362 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:42.359+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:51:42.462 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:42.456+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:42.499 INFO:teuthology.orchestra.run.smithi094.stdout:truncating image image_client.0-clone34 from 0xca8beb9 (overlap 0xca8beb9) to 0x1441312 2024-09-21T22:51:42.527 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:42.527 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:42.527 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:42.527 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:42.527 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:42.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:42.592 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:42.588+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:42.664 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:42.659+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 84.481560529999996, 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:42.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.664997532000001, 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:42.720 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.488032394000001, 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:42.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.876923738000002, 2024-09-21T22:51:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:42.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.697118539000002, 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:42.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:42.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.619498868000001, 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.619233596999997, 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:42.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.289160799999998, 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:42.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.108074850000001, 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:42.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.093459484, 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.093076893000003, 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.091064428999999, 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:42.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.062339629, 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:42.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:42.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:42.763 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.059646858999997, 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:42.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.299187235000002, 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.612265845, 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:51:42.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168816 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'545, mlcod=87'543)", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.8175230139999998, 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033834189999999999, 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:42.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891267+0000", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2500000000000001e-06 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891281+0000", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3723000000000001e-05 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891323+0000", 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2926000000000002e-05 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891344+0000", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0250000000000001e-05 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891475+0000", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013148000000000001 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891506+0000", 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0784999999999997e-05 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894584+0000", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003077598 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894648+0000", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4406999999999995e-05 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168818 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'547, mlcod=87'543)", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:42.779 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.8155626580000002, 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039584049999999999, 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893227+0000", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7460000000000001e-06 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893242+0000", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5067e-05 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893273+0000", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0920000000000002e-05 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893292+0000", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9732999999999999e-05 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893858+0000", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000565209 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893891+0000", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3822999999999998e-05 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897073+0000", 2024-09-21T22:51:42.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031815010000000002 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897183+0000", 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110406 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168827 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'552, mlcod=87'549)", 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.8080915150000001, 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003413499, 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900698+0000", 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.787e-06 2024-09-21T22:51:42.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900712+0000", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4486e-05 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900746+0000", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3330000000000001e-05 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900765+0000", 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.906e-05 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900904+0000", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139519 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900980+0000", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5636000000000006e-05 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904045+0000", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030655510000000001 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904110+0000", 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4129999999999998e-05 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:169241 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'653, mlcod=87'650)", 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.6807500189999995, 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032275400000000001, 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028047+0000", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7059999999999999e-06 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028061+0000", 2024-09-21T22:51:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.411e-05 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028094+0000", 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2292999999999997e-05 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028111+0000", 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:42.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028274+0000", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162442 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028329+0000", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5112999999999999e-05 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031115+0000", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002785624 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031265+0000", 2024-09-21T22:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150512 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:42.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:42.792 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:42.787+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:42.893 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:42.887+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:42.926 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:42.926 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:42.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:42.993 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:42.988+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:43.094 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:43.088+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:43.111 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:43.111 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:43.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:43.195 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:43.188+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:43.249 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264538321000003, 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264508319999997, 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264480614, 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264434663999999, 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264394529, 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264364530000002, 2024-09-21T22:51:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264330469000001, 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264286614, 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264250482999998, 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264199902000001, 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264094688, 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.264031178000003, 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.263932976, 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.263867713000003, 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.263801796000003, 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.263724693999997, 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.263651093999997, 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.37855017, 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.099689594000001, 2024-09-21T22:51:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0139305949999997, 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.333 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:43.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:43.334 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:43.331+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:43.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:43.435 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:43.428+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:43.529 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4382.0:171065 3.1d e92/24)", 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:43.529179+0000", 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0831000000000001e-05, 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5950000000000002e-05, 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529179+0000", 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529179+0000", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529181+0000", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1149999999999999e-06 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529188+0000", 2024-09-21T22:51:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3980000000000002e-06 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529211+0000", 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.332e-05 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529223+0000", 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1928e-05 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:171065 3.1d 3:be8d4fa2:::rbd_data.111e470eb4b3.0000000000000003:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:43.520235+0000", 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0090054160000000005, 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091176629999999998, 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 171065 2024-09-21T22:51:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520235+0000", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520235+0000", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520236+0000", 2024-09-21T22:51:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3120000000000001e-06 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520250+0000", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4059e-05 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520264+0000", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3825000000000001e-05 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.520273+0000", 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4390000000000006e-06 2024-09-21T22:51:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.523018+0000", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00274442 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.523079+0000", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1078e-05 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.523204+0000", 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012464999999999999 2024-09-21T22:51:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.526825+0000", 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036214699999999999 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.529352+0000", 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002526646 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 2 2024-09-21T22:51:43.537 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:43.538 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:43.532+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:43.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:43.566 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:43.566 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:43.586 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:43.587 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:43.605 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T22:51:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:43.624 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:43.624 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-21T22:51:43.639 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:43.635+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:43.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:43.665 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:43.665 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:43.672 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:43.672 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:43.687 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:43.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:43.708 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:43.708 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:43.777 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:43.775+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:43.840 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:43.835+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:43.856 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:43.856 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:43.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:43.941 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:43.936+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:44.015 INFO:teuthology.orchestra.run.smithi094.stdout:5933 write 0x30f944f thru 0x3104b15 (0xb6c7 bytes) 2024-09-21T22:51:44.030 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:44.030 INFO:teuthology.orchestra.run.smithi094.stdout:5935 writesame 0x9a5fb51 thru 0x9a6090d data_size 0x1(0xdbd bytes) 2024-09-21T22:51:44.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:44.040 INFO:teuthology.orchestra.run.smithi094.stdout:5936 read 0x5a24065 thru 0x5a33d29 (0xfcc5 bytes) 2024-09-21T22:51:44.041 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.041 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.041 INFO:teuthology.orchestra.run.smithi094.stdout:5938 punch from 0x5838d76 to 0x5845624, (0xc8ae bytes) 2024-09-21T22:51:44.042 INFO:teuthology.orchestra.run.smithi094.stdout:5939 write 0x5c42680 thru 0x5c467a4 (0x4125 bytes) 2024-09-21T22:51:44.042 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:44.036+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:44.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:44.058 INFO:teuthology.orchestra.run.smithi094.stdout:5940 writesame 0xe0abc11 thru 0xe0aeffc data_size 0x2(0x33ec bytes) 2024-09-21T22:51:44.063 INFO:teuthology.orchestra.run.smithi094.stdout:5941 punch from 0x5571414 to 0x5572a46, (0x1632 bytes) 2024-09-21T22:51:44.063 INFO:teuthology.orchestra.run.smithi094.stdout:5942 writesame 0xc0725d9 thru 0xc075d56 data_size 0x1(0x377e bytes) 2024-09-21T22:51:44.074 INFO:teuthology.orchestra.run.smithi094.stdout:5943 read 0x6a5e9a thru 0x6b3043 (0xd1aa bytes) 2024-09-21T22:51:44.074 INFO:teuthology.orchestra.run.smithi094.stdout:5944 read 0x11801a6 thru 0x1186ad7 (0x6932 bytes) 2024-09-21T22:51:44.075 INFO:teuthology.orchestra.run.smithi094.stdout:5946 read 0xe088d41 thru 0xe094677 (0xb937 bytes) 2024-09-21T22:51:44.080 INFO:teuthology.orchestra.run.smithi094.stdout:5947 trunc from 0xe0aeffd to 0xa42452e 2024-09-21T22:51:44.104 INFO:teuthology.orchestra.run.smithi094.stdout:5948 writesame 0x48eef18 thru 0x48effbb data_size 0x1e(0x10a4 bytes) 2024-09-21T22:51:44.106 INFO:teuthology.orchestra.run.smithi094.stdout:5949 read 0xf0a4dc thru 0xf11a3b (0x7560 bytes) 2024-09-21T22:51:44.106 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:44.119 INFO:teuthology.orchestra.run.smithi094.stdout:5951 writesame 0xc14b838 thru 0xc1532fc data_size 0x1(0x7ac5 bytes) 2024-09-21T22:51:44.119 INFO:teuthology.orchestra.run.smithi094.stdout:5955 read 0x943bb57 thru 0x9442290 (0x673a bytes) 2024-09-21T22:51:44.119 INFO:teuthology.orchestra.run.smithi094.stdout:5958 writesame 0x2a54f37 thru 0x2a5cdda data_size 0xa(0x7ea4 bytes) 2024-09-21T22:51:44.129 INFO:teuthology.orchestra.run.smithi094.stdout:5959 punch from 0x739732c to 0x739cf41, (0x5c15 bytes) 2024-09-21T22:51:44.135 INFO:teuthology.orchestra.run.smithi094.stdout:5961 trunc from 0xc1532fd to 0x8630403 2024-09-21T22:51:44.142 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:44.136+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:44.169 INFO:teuthology.orchestra.run.smithi094.stdout:5962 flatten 2024-09-21T22:51:44.169 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.169 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.187 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.187 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T22:51:44.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:44.204 INFO:teuthology.orchestra.run.smithi094.stdout:5963 write 0x57cffe6 thru 0x57d5f0b (0x5f26 bytes) 2024-09-21T22:51:44.211 INFO:teuthology.orchestra.run.smithi094.stdout:5965 trunc from 0x8630403 to 0xbcf4cce 2024-09-21T22:51:44.219 INFO:teuthology.orchestra.run.smithi094.stdout:5966 write 0x3742053 thru 0x3746a19 (0x49c7 bytes) 2024-09-21T22:51:44.221 INFO:teuthology.orchestra.run.smithi094.stdout:5968 read 0x20bf802 thru 0x20c2d8b (0x358a bytes) 2024-09-21T22:51:44.221 INFO:teuthology.orchestra.run.smithi094.stdout:5969 writesame 0x499de7 thru 0x4a0999 data_size 0x1(0x6bb3 bytes) 2024-09-21T22:51:44.239 INFO:teuthology.orchestra.run.smithi094.stdout:5971 read 0x5e37930 thru 0x5e3aa3f (0x3110 bytes) 2024-09-21T22:51:44.240 INFO:teuthology.orchestra.run.smithi094.stdout:5972 punch from 0x4efe82d to 0x4f02f4b, (0x471e bytes) 2024-09-21T22:51:44.251 INFO:teuthology.orchestra.run.smithi094.stdout:5973 trunc from 0xbcf4cce to 0x455c97b 2024-09-21T22:51:44.253 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:44.251+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:44.265 INFO:teuthology.orchestra.run.smithi094.stdout:5974 punch from 0x3f7fc94 to 0x3f8cf90, (0xd2fc bytes) 2024-09-21T22:51:44.265 INFO:teuthology.orchestra.run.smithi094.stdout:5976 punch from 0x38d5053 to 0x38dc91d, (0x78ca bytes) 2024-09-21T22:51:44.282 INFO:teuthology.orchestra.run.smithi094.stdout:5977 writesame 0xa58c484 thru 0xa58d5c7 data_size 0x14(0x1144 bytes) 2024-09-21T22:51:44.282 INFO:teuthology.orchestra.run.smithi094.stdout:5978 read 0xa5499cc thru 0xa5539bb (0x9ff0 bytes) 2024-09-21T22:51:44.282 INFO:teuthology.orchestra.run.smithi094.stdout:5979 read 0x7b160e4 thru 0x7b1c613 (0x6530 bytes) 2024-09-21T22:51:44.282 INFO:teuthology.orchestra.run.smithi094.stdout:5980 write 0x1fa1f5b thru 0x1fa9b83 (0x7c29 bytes) 2024-09-21T22:51:44.284 INFO:teuthology.orchestra.run.smithi094.stdout:5981 read 0xe57539 thru 0xe61be7 (0xa6af bytes) 2024-09-21T22:51:44.285 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:44.285 INFO:teuthology.orchestra.run.smithi094.stdout:5983 read 0x24f8af5 thru 0x250414f (0xb65b bytes) 2024-09-21T22:51:44.285 INFO:teuthology.orchestra.run.smithi094.stdout:5985 read 0x8d48970 thru 0x8d4ce66 (0x44f7 bytes) 2024-09-21T22:51:44.289 INFO:teuthology.orchestra.run.smithi094.stdout:5986 trunc from 0xa58d5c8 to 0x7e8f3e9 2024-09-21T22:51:44.301 INFO:teuthology.orchestra.run.smithi094.stdout:5989 trunc from 0x7e8f3e9 to 0xbfcc271 2024-09-21T22:51:44.309 INFO:teuthology.orchestra.run.smithi094.stdout:5991 writesame 0x803927a thru 0x8042f00 data_size 0x39(0x9c87 bytes) 2024-09-21T22:51:44.314 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-21T22:51:44.315 INFO:teuthology.orchestra.run.smithi094.stdout:5994 trunc from 0xbfcc271 to 0xb390e4c 2024-09-21T22:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.333 INFO:teuthology.orchestra.run.smithi094.stdout:5995 write 0xbb22ae8 thru 0xbb2be7d (0x9396 bytes) 2024-09-21T22:51:44.339 INFO:teuthology.orchestra.run.smithi094.stdout:5996 writesame 0x58ea02c thru 0x58f2034 data_size 0x1(0x8009 bytes) 2024-09-21T22:51:44.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:44.344 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:44.336+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:44.360 INFO:teuthology.orchestra.run.smithi094.stdout:5999 read 0x303913f thru 0x304569d (0xc55f bytes) 2024-09-21T22:51:44.369 INFO:teuthology.orchestra.run.smithi094.stdout:6000 writesame 0xcd64e58 thru 0xcd6605c data_size 0x1(0x1205 bytes) 2024-09-21T22:51:44.397 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T22:51:44.444 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:44.440+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:44.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:51:44.519 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.519 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:44.581 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:44.579+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:44.645 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:44.640+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:44.660 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.660 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.674 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:44.761 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:44.759+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:44.834 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.834 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.847 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:44.843+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:44.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:44.948 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:44.943+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:44.975 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:44.976 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:44.990 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:45.050 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:45.044+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:45.106 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:45.107 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:45.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:45.165 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:45.160+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:45.252 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:45.251+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:45.289 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:45.289 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:45.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:45.352 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:45.348+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:45.423 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:45.423 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:45.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:45.452 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:45.447+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T22:51:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:51:45.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T22:51:45.588 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:45.584+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:45.594 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:45.594 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:45.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:45.653 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:45.647+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:45.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:45.754 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:45.751+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:45.768 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:45.768 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:45.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:45.854 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:45.851+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:45.875 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:45.875 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:45.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:45.892 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:45.909 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:45.925 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:45.926 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:45.945 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:45.945 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:45.945 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:45.945 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:45.946 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:45.955 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:45.948+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:45.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:45.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:45.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:45.989 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:46.011 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:46.056 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:46.051+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:46.085 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:46.086 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:46.086 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:46.086 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:46.086 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:46.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:46.156 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:46.155+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:46.227 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:46.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.136528349999999, 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:46.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.736328485000001, 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:46.232 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:46.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734814225000001, 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:46.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734760923000003, 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:46.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:46.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734705875000003, 2024-09-21T22:51:46.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:46.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734636690999999, 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734574123999998, 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.254 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:46.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:46.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734524729999997, 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:46.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.734020946999998, 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.266 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733953368000002, 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:46.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:46.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733404125, 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:46.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:46.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:46.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733326644000002, 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:46.275 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733162513000003, 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:46.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:46.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:46.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733101533999999, 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:46.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:46.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733057922, 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.733013440000001, 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:46.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.732968272000001, 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.732925975000001, 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.247493386000002, 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:46.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.895998912, 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:51:46.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:51:46.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.315 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:46.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:46.317 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:46.315+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:46.417 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:46.412+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:46.443 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:46.443 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:46.443 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:46.443 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:46.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T22:51:46.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T22:51:46.518 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:46.515+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:46.603 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:46.603 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:46.603 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:46.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:46.620 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:46.615+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:46.722 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:46.716+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.505036971999999, 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.688473974000004, 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.739 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:46.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.511508835999997, 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:46.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.900400179999998, 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.749 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:46.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:46.754 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.720594980999998, 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:46.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.642975309999997, 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:46.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:46.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:46.763 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.642710039000001, 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:46.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:46.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.312637242000001, 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.131551291999997, 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.116935925999996, 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.116553334999999, 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.114540871000003, 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.085816070999996, 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.083123301000001, 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.322663677000001, 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.635742286999999, 2024-09-21T22:51:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:51:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168816 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'545, mlcod=87'543)", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8409994560000005, 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033834189999999999, 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891267+0000", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2500000000000001e-06 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891281+0000", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3723000000000001e-05 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891323+0000", 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2926000000000002e-05 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891344+0000", 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0250000000000001e-05 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891475+0000", 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013148000000000001 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891506+0000", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0784999999999997e-05 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894584+0000", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003077598 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894648+0000", 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4406999999999995e-05 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168818 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'547, mlcod=87'543)", 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8390391000000008, 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039584049999999999, 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893227+0000", 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7460000000000001e-06 2024-09-21T22:51:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893242+0000", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5067e-05 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893273+0000", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0920000000000002e-05 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893292+0000", 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9732999999999999e-05 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893858+0000", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000565209 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893891+0000", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3822999999999998e-05 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897073+0000", 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031815010000000002 2024-09-21T22:51:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897183+0000", 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110406 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168827 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'552, mlcod=87'549)", 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8315679570000007, 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003413499, 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900698+0000", 2024-09-21T22:51:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.787e-06 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900712+0000", 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4486e-05 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900746+0000", 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3330000000000001e-05 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900765+0000", 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.906e-05 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900904+0000", 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139519 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900980+0000", 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5636000000000006e-05 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904045+0000", 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030655510000000001 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904110+0000", 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4129999999999998e-05 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:169241 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'653, mlcod=87'650)", 2024-09-21T22:51:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.7042264609999993, 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032275400000000001, 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028047+0000", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7059999999999999e-06 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028061+0000", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.411e-05 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028094+0000", 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2292999999999997e-05 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028111+0000", 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028274+0000", 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162442 2024-09-21T22:51:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028329+0000", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5112999999999999e-05 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031115+0000", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002785624 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031265+0000", 2024-09-21T22:51:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150512 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:46.824 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:46.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:46.826 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:46.820+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:46.926 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:46.923+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:46.958 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:46.958 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:46.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:47.028 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:47.024+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:47.128 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:47.128+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:47.135 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:47.135 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:47.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:47.229 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:47.228+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290229867999997, 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290199866999998, 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290172161000001, 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:47.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:47.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290126211, 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:47.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:47.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:47.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290086075999994, 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:47.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:47.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290056077000003, 2024-09-21T22:51:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:47.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:47.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.290022015999995, 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:47.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:47.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289978160999993, 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289942030000006, 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:47.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:47.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289891448999995, 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:47.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:47.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289786234999994, 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289722725000004, 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:47.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:47.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289624523000001, 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:47.324 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:47.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:47.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289559260000004, 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:47.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:47.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289493343000004, 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:47.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:47.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289416240999998, 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.289342641000005, 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.404241717000001, 2024-09-21T22:51:47.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.125381140999998, 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:47.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.039622142000001, 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:51:47.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:51:47.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.360 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:47.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:47.361 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:47.360+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:47.462 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:47.459+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T22:51:47.490 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:47.491 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:47.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:51:47.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:47.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T22:51:47.597 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:47.595+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:47.663 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:47.656+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:47.689 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:47.689 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:47.703 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:47.789 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:47.787+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:47.864 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:47.860+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588683168000003, 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588641322000001, 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:47.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588595847999997, 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588560919999999, 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:47.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.888 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588528674000003, 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:47.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588492942000002, 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.894 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:47.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:47.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588455766000003, 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:47.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.588421873000001, 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.579986528999999, 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910969796, 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:51:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910918176999999, 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910866874, 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:51:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910808958000001, 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:51:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910753299, 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:51:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910697150000001, 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910634657999999, 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:51:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:51:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:51:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:51:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910533839999999, 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:51:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:51:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910478020999999, 2024-09-21T22:51:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:51:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.910425315000001, 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.362331642, 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:51:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:51:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:51:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:51:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:47.966 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:47.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:47.968 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:47.964+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:48.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:48.068 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:48.063+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:48.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:48.131 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:48.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:48.179 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:48.179 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:48.190 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:48.187+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:48.195 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:48.195 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:48.212 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:48.212 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:48.228 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:48.228 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:48.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:48.246 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:48.266 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:48.269 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:48.267+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:48.287 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:48.287 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:48.305 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:48.309 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:48.309 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:48.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:48.414 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:48.411+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:48.471 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:48.468+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:48.484 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:51:48.488 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T22:51:48.492 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:48.492 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:48.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:48.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T22:51:48.594 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:48.591+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:48.673 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:48.671+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:48.683 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:48.683 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:48.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:48.790 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:48.787+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:48.866 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:48.866 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:48.875 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:48.872+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:48.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:48.976 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:48.971+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:49.046 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.046 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:49.102 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:49.099+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:49.178 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:49.172+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:49.181 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.182 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:49.284 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:49.284+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:49.355 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.355 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:49.412 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:49.412+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:49.482 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:49.479+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:49.496 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T22:51:49.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T22:51:49.541 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.542 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:49.583 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:49.579+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:49.684 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:49.679+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:49.725 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.726 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:49.785 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:49.783+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:49.845 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.846 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:49.904 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:49.904+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:49.972 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:49.972 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:49.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:49.988 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:49.983+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:50.089 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:50.084+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:50.155 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:50.155 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:50.169 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:50.212 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:50.208+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:50.291 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:50.287+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:50.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:50.323 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:50.323 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:50.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:50.394 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:50.387+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:50.459 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:50.459 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:50.476 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:50.476 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:50.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:50.493 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:50.487+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:50.493 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:50.503 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T22:51:50.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:51:50.522 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:50.522 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.432587525000002, 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:51:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.032387659999998, 2024-09-21T22:51:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:51:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:51:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030873400000004, 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030820098000007, 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:51:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:51:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030765049999999, 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:51:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:51:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030695866000002, 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030633299000002, 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:51:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:51:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:51:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030583905, 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:51:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:51:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:51:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030080122000001, 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:51:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:51:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:51:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.030012542999998, 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:51:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029463300000003, 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029385818999998, 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029221688000007, 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029160708999996, 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:51:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:51:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029117096999997, 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:51:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029072615000004, 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:51:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.029027447000004, 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:51:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.028985149999997, 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:51:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:51:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:51:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:51:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.543552560999998, 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.192058086999999, 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:51:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:50.607 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:50.608 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:50.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T22:51:50.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:50.610 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:50.610 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:50.610 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:50.611 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:50.611 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:50.611 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:50.611 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:50.604+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:50.617 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:50.618 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:50.709 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:50.704+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:50.733 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:50.733 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:50.734 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:51:50.734 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:50.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:50.810 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:50.807+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:50.897 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:50.897 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:51:50.897 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:51:50.897 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:51:50.898 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:50.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:50.911 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:50.904+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:51.012 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:51.008+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:51.072 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:51.072 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:51.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:51.072 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:51.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.842143897, 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:51.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:51.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.025580899000005, 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:51.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.848615761000005, 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:51.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:51.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.237507104999999, 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.057701905999998, 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:51.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.980082234999998, 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:51.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.979816964000001, 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:51.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.649744167000001, 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:51.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:51.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:51.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.468658216999998, 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:51.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:51.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:51.109 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.454042850999997, 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:51.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:51.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.453660259999999, 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.451647796000003, 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.422922995999997, 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.420230226000001, 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:51.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:51.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:51.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.659770602000002, 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:51.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:51.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.972849212, 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:51:51.134 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168816 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'545, mlcod=87'543)", 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.178106380999999, 2024-09-21T22:51:51.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033834189999999999, 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891267+0000", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2500000000000001e-06 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891281+0000", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3723000000000001e-05 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891323+0000", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2926000000000002e-05 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891344+0000", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0250000000000001e-05 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891475+0000", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013148000000000001 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891506+0000", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0784999999999997e-05 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894584+0000", 2024-09-21T22:51:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003077598 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894648+0000", 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4406999999999995e-05 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168818 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'547, mlcod=87'543)", 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.176146025, 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039584049999999999, 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893227+0000", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7460000000000001e-06 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893242+0000", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5067e-05 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893273+0000", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0920000000000002e-05 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893292+0000", 2024-09-21T22:51:51.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9732999999999999e-05 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893858+0000", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000565209 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893891+0000", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3822999999999998e-05 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897073+0000", 2024-09-21T22:51:51.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031815010000000002 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897183+0000", 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110406 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168827 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'552, mlcod=87'549)", 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.168674881999999, 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003413499, 2024-09-21T22:51:51.146 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900698+0000", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.787e-06 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900712+0000", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4486e-05 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900746+0000", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3330000000000001e-05 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900765+0000", 2024-09-21T22:51:51.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.906e-05 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900904+0000", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139519 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900980+0000", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5636000000000006e-05 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904045+0000", 2024-09-21T22:51:51.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030655510000000001 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904110+0000", 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4129999999999998e-05 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:169241 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'653, mlcod=87'650)", 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.041333386, 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032275400000000001, 2024-09-21T22:51:51.150 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:51.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028047+0000", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7059999999999999e-06 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028061+0000", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.411e-05 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028094+0000", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2292999999999997e-05 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028111+0000", 2024-09-21T22:51:51.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028274+0000", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162442 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028329+0000", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5112999999999999e-05 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031115+0000", 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002785624 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:51.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031265+0000", 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150512 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:51.154 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:51.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:51.156 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:51.152+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:51.256 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:51.256+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:51.288 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:51.288 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:51.289 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:51.289 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:51.303 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:51.357 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:51.356+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:51.425 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:51.426 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:51.426 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:51.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:51.486 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:51.483+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:51:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T22:51:51.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589718286999997, 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589688285999998, 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.58966058, 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.58961463, 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589574494999994, 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589544496000002, 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589510434999994, 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589466580000007, 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589430449000005, 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589379867999995, 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589274653999993, 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589211144000004, 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589112942, 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:51.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.589047679000004, 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.588981762000003, 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.588904659999997, 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.588831060000004, 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.703730136000001, 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.424869560000001, 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:51:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.339110560999998, 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:51:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:51.658 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:51.658 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:51.584+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:51.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:51.660 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:51.655+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:51.759 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:51.756+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:51.825 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:51.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:51.825 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:51.825 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:51.838 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:51.860 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:51.860+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:51.961 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:51.955+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:51.961 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:51.962 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:51.962 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:51.962 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:51.962 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:51.976 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:52.062 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:52.059+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866962352000002, 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866920506, 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866875032000003, 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866840103999998, 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866807858000001, 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866772126000001, 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866734950000001, 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.866701057, 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.858265713000002, 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.189248979999999, 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.189197361000002, 2024-09-21T22:51:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:51:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:51:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.189146057999999, 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:51:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:51:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.189088141999999, 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:51:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:51:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:51:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.189032482999998, 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:51:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:51:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.188976333999999, 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:51:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:51:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.188913842000002, 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:51:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.188813024000002, 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:51:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:51:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.188757205000002, 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.188704499, 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:51:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6406108259999996, 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:51:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:51:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.238 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:52.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:52.240 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:52.235+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:52.340 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:52.335+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:52.407 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:52.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:52.466 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:52.463+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:52.509 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T22:51:52.538 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T22:51:52.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:51:52.543 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:52.539+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:52.598 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:52.599 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:52.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:52.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:52.662 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:52.659+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:52.745 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:52.744+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:52.787 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:52.787 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 119.856204253, 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:52.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.311132393999998, 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:52.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.311068943999999, 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:52.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:52.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.311005765999994, 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310949425999993, 2024-09-21T22:51:52.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:52.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:52.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310907684, 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:52.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310847082999999, 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310766646999994, 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310690536999999, 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:52.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:52.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310603463999996, 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:52.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:52.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310559975999993, 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310526338000003, 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:52.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:52.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:52.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310492572000001, 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:52.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:52.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310461287999999, 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:52.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310415933000002, 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310384436999996, 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.310349656, 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.292343080999999, 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.969762930999998, 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:51:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.268324742000001, 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:52.882 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:52.883 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:52.883 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:52.884 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:52.884 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:52.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:52.885 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:52.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:52.886 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:52.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:52.886 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:52.887 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:52.880+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:52.896 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:51:52.917 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:52.917 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:52.921 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-21T22:51:52.509618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":713373465,"num_objects":2359,"num_object_clones":675,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":1123,"num_read":212968,"num_read_kb":710289,"num_write":22876,"num_write_kb":333955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18562,"ondisk_log_size":18562,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":811644,"kb_used_data":451636,"kb_used_omap":55,"kb_used_meta":359560,"kb_avail":754163076,"statfs":{"total":773094113280,"available":772262989824,"internally_reserved":0,"allocated":462475264,"data_stored":506486835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":368189802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-203293705,"num_objects":-86,"num_object_clones":-25,"num_object_copies":-172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-86,"num_whiteouts":-11,"num_read":863,"num_read_kb":733,"num_write":186,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":5834135,"num_keys_recovered":12,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020946"},"pg_stats":[{"pgid":"3.1f","version":"96'921","reported_seq":6926,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497725+0000","last_change":"2024-09-21T22:51:49.498031+0000","last_active":"2024-09-21T22:51:49.497725+0000","last_peered":"2024-09-21T22:51:49.497725+0000","last_clean":"2024-09-21T22:51:49.497725+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:51:49.497725+0000","last_undegraded":"2024-09-21T22:51:49.497725+0000","last_fullsized":"2024-09-21T22:51:49.497725+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:18:38.222143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014314040000000001,"stat_sum":{"num_bytes":31556376,"num_objects":83,"num_object_clones":26,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":35,"num_read":9488,"num_read_kb":21669,"num_write":1212,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1e","version":"95'479","reported_seq":5818,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497401+0000","last_change":"2024-09-21T22:51:49.497672+0000","last_active":"2024-09-21T22:51:49.497401+0000","last_peered":"2024-09-21T22:51:49.497401+0000","last_clean":"2024-09-21T22:51:49.497401+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:51:49.497401+0000","last_undegraded":"2024-09-21T22:51:49.497401+0000","last_fullsized":"2024-09-21T22:51:49.497401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:04:28.748600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272022,"stat_sum":{"num_bytes":25291019,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":43,"num_read":7105,"num_read_kb":20265,"num_write":451,"num_write_kb":5427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1d","version":"99'970","reported_seq":6999,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.522498+0000","last_change":"2024-09-21T22:51:50.522787+0000","last_active":"2024-09-21T22:51:50.522498+0000","last_peered":"2024-09-21T22:51:50.522498+0000","last_clean":"2024-09-21T22:51:50.522498+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:51:50.522498+0000","last_undegraded":"2024-09-21T22:51:50.522498+0000","last_fullsized":"2024-09-21T22:51:50.522498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:56.386675+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013660189999999999,"stat_sum":{"num_bytes":29531025,"num_objects":77,"num_object_clones":18,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":37,"num_read":10236,"num_read_kb":47062,"num_write":1452,"num_write_kb":18437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.1c","version":"99'756","reported_seq":1983,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.562402+0000","last_change":"2024-09-21T22:51:50.509123+0000","last_active":"2024-09-21T22:51:50.562402+0000","last_peered":"2024-09-21T22:51:50.562402+0000","last_clean":"2024-09-21T22:51:50.562402+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:51:50.562402+0000","last_undegraded":"2024-09-21T22:51:50.562402+0000","last_fullsized":"2024-09-21T22:51:50.562402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:20.773179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013178599999999999,"stat_sum":{"num_bytes":8102632,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":37,"num_read":3471,"num_read_kb":10834,"num_write":1023,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.1b","version":"96'1007","reported_seq":6290,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495592+0000","last_change":"2024-09-21T22:51:49.496285+0000","last_active":"2024-09-21T22:51:49.495592+0000","last_peered":"2024-09-21T22:51:49.495592+0000","last_clean":"2024-09-21T22:51:49.495592+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:51:49.495592+0000","last_undegraded":"2024-09-21T22:51:49.495592+0000","last_fullsized":"2024-09-21T22:51:49.495592+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:39.587519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011844990000000001,"stat_sum":{"num_bytes":29086151,"num_objects":71,"num_object_clones":25,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":9424,"num_read_kb":37686,"num_write":1574,"num_write_kb":12277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1a","version":"98'735","reported_seq":8260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.708618+0000","last_change":"2024-09-21T22:51:49.496765+0000","last_active":"2024-09-21T22:51:49.708618+0000","last_peered":"2024-09-21T22:51:49.708618+0000","last_clean":"2024-09-21T22:51:49.708618+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:51:49.708618+0000","last_undegraded":"2024-09-21T22:51:49.708618+0000","last_fullsized":"2024-09-21T22:51:49.708618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:42:35.320223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010989179999999999,"stat_sum":{"num_bytes":13332042,"num_objects":73,"num_object_clones":25,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":9793,"num_read_kb":12962,"num_write":1085,"num_write_kb":4809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.19","version":"98'474","reported_seq":8347,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651773+0000","last_change":"2024-09-21T22:51:49.651773+0000","last_active":"2024-09-21T22:51:49.651773+0000","last_peered":"2024-09-21T22:51:49.651773+0000","last_clean":"2024-09-21T22:51:49.651773+0000","last_became_active":"2024-09-21T22:51:42.461781+0000","last_became_peered":"2024-09-21T22:51:42.461781+0000","last_unstale":"2024-09-21T22:51:49.651773+0000","last_undegraded":"2024-09-21T22:51:49.651773+0000","last_fullsized":"2024-09-21T22:51:49.651773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:35:10.224125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013902039999999999,"stat_sum":{"num_bytes":39383560,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":8664,"num_read_kb":60910,"num_write":491,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":20755516,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.18","version":"87'1019","reported_seq":11056,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494610+0000","last_change":"2024-09-21T22:51:49.496744+0000","last_active":"2024-09-21T22:51:49.494610+0000","last_peered":"2024-09-21T22:51:49.494610+0000","last_clean":"2024-09-21T22:51:49.494610+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:51:49.494610+0000","last_undegraded":"2024-09-21T22:51:49.494610+0000","last_fullsized":"2024-09-21T22:51:49.494610+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:44:22.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025291580000000001,"stat_sum":{"num_bytes":23998665,"num_objects":80,"num_object_clones":23,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":14180,"num_read_kb":27547,"num_write":1639,"num_write_kb":11372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.17","version":"98'414","reported_seq":3030,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.711868+0000","last_change":"2024-09-21T22:51:49.496754+0000","last_active":"2024-09-21T22:51:49.711868+0000","last_peered":"2024-09-21T22:51:49.711868+0000","last_clean":"2024-09-21T22:51:49.711868+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:51:49.711868+0000","last_undegraded":"2024-09-21T22:51:49.711868+0000","last_fullsized":"2024-09-21T22:51:49.711868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:52.369390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015310020000000001,"stat_sum":{"num_bytes":6684916,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":3539,"num_read_kb":6160,"num_write":395,"num_write_kb":3436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.16","version":"93'380","reported_seq":8436,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.522918+0000","last_change":"2024-09-21T22:51:50.523156+0000","last_active":"2024-09-21T22:51:50.522918+0000","last_peered":"2024-09-21T22:51:50.522918+0000","last_clean":"2024-09-21T22:51:50.522918+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:51:50.522918+0000","last_undegraded":"2024-09-21T22:51:50.522918+0000","last_fullsized":"2024-09-21T22:51:50.522918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:08.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012721846,"stat_sum":{"num_bytes":17639789,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":40,"num_read":7939,"num_read_kb":9321,"num_write":332,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.15","version":"95'407","reported_seq":11093,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.608927+0000","last_change":"2024-09-21T22:51:49.609657+0000","last_active":"2024-09-21T22:51:49.608927+0000","last_peered":"2024-09-21T22:51:49.608927+0000","last_clean":"2024-09-21T22:51:49.608927+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:51:49.608927+0000","last_undegraded":"2024-09-21T22:51:49.608927+0000","last_fullsized":"2024-09-21T22:51:49.608927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:48:02.236052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018147949999999999,"stat_sum":{"num_bytes":40696842,"num_objects":79,"num_object_clones":19,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":11080,"num_read_kb":42972,"num_write":427,"num_write_kb":18958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.14","version":"96'429","reported_seq":9165,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.496357+0000","last_change":"2024-09-21T22:51:49.497022+0000","last_active":"2024-09-21T22:51:49.496357+0000","last_peered":"2024-09-21T22:51:49.496357+0000","last_clean":"2024-09-21T22:51:49.496357+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:51:49.496357+0000","last_undegraded":"2024-09-21T22:51:49.496357+0000","last_fullsized":"2024-09-21T22:51:49.496357+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:37:51.248758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011734429999999999,"stat_sum":{"num_bytes":24498538,"num_objects":71,"num_object_clones":20,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":34,"num_read":9237,"num_read_kb":22032,"num_write":505,"num_write_kb":15782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.515676+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:51:51.515676+0000","last_peered":"2024-09-21T22:51:51.515676+0000","last_clean":"2024-09-21T22:51:51.515676+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:51:51.515676+0000","last_undegraded":"2024-09-21T22:51:51.515676+0000","last_fullsized":"2024-09-21T22:51:51.515676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'508","reported_seq":6411,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495741+0000","last_change":"2024-09-21T22:51:49.495741+0000","last_active":"2024-09-21T22:51:49.495741+0000","last_peered":"2024-09-21T22:51:49.495741+0000","last_clean":"2024-09-21T22:51:49.495741+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:51:49.495741+0000","last_undegraded":"2024-09-21T22:51:49.495741+0000","last_fullsized":"2024-09-21T22:51:49.495741+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:57.618096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096816600000000001,"stat_sum":{"num_bytes":18207663,"num_objects":70,"num_object_clones":20,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":5985,"num_read_kb":22287,"num_write":501,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.650382+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:51:49.650382+0000","last_peered":"2024-09-21T22:51:49.650382+0000","last_clean":"2024-09-21T22:51:49.650382+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:51:49.650382+0000","last_undegraded":"2024-09-21T22:51:49.650382+0000","last_fullsized":"2024-09-21T22:51:49.650382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"98'439","reported_seq":4089,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.498036+0000","last_change":"2024-09-21T22:51:49.498036+0000","last_active":"2024-09-21T22:51:49.498036+0000","last_peered":"2024-09-21T22:51:49.498036+0000","last_clean":"2024-09-21T22:51:49.498036+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:51:49.498036+0000","last_undegraded":"2024-09-21T22:51:49.498036+0000","last_fullsized":"2024-09-21T22:51:49.498036+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:50.919666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013130559999999999,"stat_sum":{"num_bytes":7934304,"num_objects":75,"num_object_clones":23,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":38,"num_read":4761,"num_read_kb":6226,"num_write":503,"num_write_kb":3540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.608075+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:51:49.608075+0000","last_peered":"2024-09-21T22:51:49.608075+0000","last_clean":"2024-09-21T22:51:49.608075+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:51:49.608075+0000","last_undegraded":"2024-09-21T22:51:49.608075+0000","last_fullsized":"2024-09-21T22:51:49.608075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"85'538","reported_seq":2461,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651833+0000","last_change":"2024-09-21T22:51:49.652184+0000","last_active":"2024-09-21T22:51:49.651833+0000","last_peered":"2024-09-21T22:51:49.651833+0000","last_clean":"2024-09-21T22:51:49.651833+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:51:49.651833+0000","last_undegraded":"2024-09-21T22:51:49.651833+0000","last_fullsized":"2024-09-21T22:51:49.651833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:29.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017327099999999999,"stat_sum":{"num_bytes":11458158,"num_objects":80,"num_object_clones":27,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":37,"num_read":2835,"num_read_kb":6413,"num_write":661,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.650589+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:51:49.650589+0000","last_peered":"2024-09-21T22:51:49.650589+0000","last_clean":"2024-09-21T22:51:49.650589+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:51:49.650589+0000","last_undegraded":"2024-09-21T22:51:49.650589+0000","last_fullsized":"2024-09-21T22:51:49.650589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"99'535","reported_seq":2499,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.527180+0000","last_change":"2024-09-21T22:51:50.527824+0000","last_active":"2024-09-21T22:51:50.527180+0000","last_peered":"2024-09-21T22:51:50.527180+0000","last_clean":"2024-09-21T22:51:50.527180+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:51:50.527180+0000","last_undegraded":"2024-09-21T22:51:50.527180+0000","last_fullsized":"2024-09-21T22:51:50.527180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:34:22.521298+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018022554,"stat_sum":{"num_bytes":9171086,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":29,"num_read":3525,"num_read_kb":13902,"num_write":533,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.3","version":"98'533","reported_seq":6161,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.679507+0000","last_change":"2024-09-21T22:51:49.499945+0000","last_active":"2024-09-21T22:51:49.679507+0000","last_peered":"2024-09-21T22:51:49.679507+0000","last_clean":"2024-09-21T22:51:49.679507+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:51:49.679507+0000","last_undegraded":"2024-09-21T22:51:49.679507+0000","last_fullsized":"2024-09-21T22:51:49.679507+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:39:09.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034278889999999999,"stat_sum":{"num_bytes":16579695,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":7810,"num_read_kb":13271,"num_write":510,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.2","version":"23'2","reported_seq":173,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.511119+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:51:50.511119+0000","last_peered":"2024-09-21T22:51:50.511119+0000","last_clean":"2024-09-21T22:51:50.511119+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:51:50.511119+0000","last_undegraded":"2024-09-21T22:51:50.511119+0000","last_fullsized":"2024-09-21T22:51:50.511119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"85'578","reported_seq":4537,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651610+0000","last_change":"2024-09-21T22:51:49.651984+0000","last_active":"2024-09-21T22:51:49.651610+0000","last_peered":"2024-09-21T22:51:49.651610+0000","last_clean":"2024-09-21T22:51:49.651610+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:51:49.651610+0000","last_undegraded":"2024-09-21T22:51:49.651610+0000","last_fullsized":"2024-09-21T22:51:49.651610+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:38.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015154140000000001,"stat_sum":{"num_bytes":25917914,"num_objects":84,"num_object_clones":27,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":37,"num_read":5773,"num_read_kb":19270,"num_write":873,"num_write_kb":12098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":176,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.136075+0000","last_change":"2024-09-21T22:51:42.453022+0000","last_active":"2024-09-21T22:51:50.136075+0000","last_peered":"2024-09-21T22:51:50.136075+0000","last_clean":"2024-09-21T22:51:50.136075+0000","last_became_active":"2024-09-21T22:51:42.452732+0000","last_became_peered":"2024-09-21T22:51:42.452732+0000","last_unstale":"2024-09-21T22:51:50.136075+0000","last_undegraded":"2024-09-21T22:51:50.136075+0000","last_fullsized":"2024-09-21T22:51:50.136075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:04.156384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"95'341","reported_seq":5505,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494544+0000","last_change":"2024-09-21T22:51:49.495336+0000","last_active":"2024-09-21T22:51:49.494544+0000","last_peered":"2024-09-21T22:51:49.494544+0000","last_clean":"2024-09-21T22:51:49.494544+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:51:49.494544+0000","last_undegraded":"2024-09-21T22:51:49.494544+0000","last_fullsized":"2024-09-21T22:51:49.494544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:50:01.343913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011115140000000001,"stat_sum":{"num_bytes":38151644,"num_objects":83,"num_object_clones":24,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":37,"num_read":5111,"num_read_kb":47271,"num_write":305,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.606762+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:51:49.606762+0000","last_peered":"2024-09-21T22:51:49.606762+0000","last_clean":"2024-09-21T22:51:49.606762+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:51:49.606762+0000","last_undegraded":"2024-09-21T22:51:49.606762+0000","last_fullsized":"2024-09-21T22:51:49.606762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'638","reported_seq":2806,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497316+0000","last_change":"2024-09-21T22:51:49.497651+0000","last_active":"2024-09-21T22:51:49.497316+0000","last_peered":"2024-09-21T22:51:49.497316+0000","last_clean":"2024-09-21T22:51:49.497316+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:51:49.497316+0000","last_undegraded":"2024-09-21T22:51:49.497316+0000","last_fullsized":"2024-09-21T22:51:49.497316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:43.085622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574935,"stat_sum":{"num_bytes":18088482,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":3750,"num_read_kb":14069,"num_write":812,"num_write_kb":10973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.3","version":"21'1","reported_seq":172,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.507593+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:51:50.507593+0000","last_peered":"2024-09-21T22:51:50.507593+0000","last_clean":"2024-09-21T22:51:50.507593+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:51:50.507593+0000","last_undegraded":"2024-09-21T22:51:50.507593+0000","last_fullsized":"2024-09-21T22:51:50.507593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.607030+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:51:49.607030+0000","last_peered":"2024-09-21T22:51:49.607030+0000","last_clean":"2024-09-21T22:51:49.607030+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:51:49.607030+0000","last_undegraded":"2024-09-21T22:51:49.607030+0000","last_fullsized":"2024-09-21T22:51:49.607030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"97'692","reported_seq":2698,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495771+0000","last_change":"2024-09-21T22:51:49.496992+0000","last_active":"2024-09-21T22:51:49.495771+0000","last_peered":"2024-09-21T22:51:49.495771+0000","last_clean":"2024-09-21T22:51:49.495771+0000","last_became_active":"2024-09-21T22:51:45.471804+0000","last_became_peered":"2024-09-21T22:51:45.471804+0000","last_unstale":"2024-09-21T22:51:49.495771+0000","last_undegraded":"2024-09-21T22:51:49.495771+0000","last_fullsized":"2024-09-21T22:51:49.495771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:51:25.842675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601253,"stat_sum":{"num_bytes":14493799,"num_objects":73,"num_object_clones":19,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":4140,"num_read_kb":7282,"num_write":936,"num_write_kb":5004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1097517,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.9","version":"97'765","reported_seq":5428,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497463+0000","last_change":"2024-09-21T22:51:49.498064+0000","last_active":"2024-09-21T22:51:49.497463+0000","last_peered":"2024-09-21T22:51:49.497463+0000","last_clean":"2024-09-21T22:51:49.497463+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:51:49.497463+0000","last_undegraded":"2024-09-21T22:51:49.497463+0000","last_fullsized":"2024-09-21T22:51:49.497463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:34:04.934264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001891053,"stat_sum":{"num_bytes":7674582,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":38,"num_read":6837,"num_read_kb":10243,"num_write":973,"num_write_kb":5643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.a","version":"100'298","reported_seq":1247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.596691+0000","last_change":"2024-09-21T22:51:51.516319+0000","last_active":"2024-09-21T22:51:51.596691+0000","last_peered":"2024-09-21T22:51:51.596691+0000","last_clean":"2024-09-21T22:51:51.596691+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:51:51.596691+0000","last_undegraded":"2024-09-21T22:51:51.596691+0000","last_fullsized":"2024-09-21T22:51:51.596691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:38.103227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057854900000000003,"stat_sum":{"num_bytes":13629912,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":891,"num_read_kb":14115,"num_write":288,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"6"}]},{"pgid":"3.b","version":"85'317","reported_seq":1391,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.496839+0000","last_change":"2024-09-21T22:51:49.497188+0000","last_active":"2024-09-21T22:51:49.496839+0000","last_peered":"2024-09-21T22:51:49.496839+0000","last_clean":"2024-09-21T22:51:49.496839+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:51:49.496839+0000","last_undegraded":"2024-09-21T22:51:49.496839+0000","last_fullsized":"2024-09-21T22:51:49.496839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:49.354546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001208277,"stat_sum":{"num_bytes":24945951,"num_objects":79,"num_object_clones":25,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":36,"num_read":1415,"num_read_kb":18056,"num_write":315,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.c","version":"99'290","reported_seq":5163,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.521378+0000","last_change":"2024-09-21T22:51:50.521810+0000","last_active":"2024-09-21T22:51:50.521378+0000","last_peered":"2024-09-21T22:51:50.521378+0000","last_clean":"2024-09-21T22:51:50.521378+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:51:50.521378+0000","last_undegraded":"2024-09-21T22:51:50.521378+0000","last_fullsized":"2024-09-21T22:51:50.521378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:35.479805+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013250288000000001,"stat_sum":{"num_bytes":19503041,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":4881,"num_read_kb":18947,"num_write":309,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.d","version":"97'569","reported_seq":3300,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.609077+0000","last_change":"2024-09-21T22:51:49.610360+0000","last_active":"2024-09-21T22:51:49.609077+0000","last_peered":"2024-09-21T22:51:49.609077+0000","last_clean":"2024-09-21T22:51:49.609077+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:51:49.609077+0000","last_undegraded":"2024-09-21T22:51:49.609077+0000","last_fullsized":"2024-09-21T22:51:49.609077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:41.619652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024746120000000002,"stat_sum":{"num_bytes":7892086,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":36,"num_read":5259,"num_read_kb":8398,"num_write":537,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.e","version":"96'575","reported_seq":6196,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.609791+0000","last_change":"2024-09-21T22:51:49.610374+0000","last_active":"2024-09-21T22:51:49.609791+0000","last_peered":"2024-09-21T22:51:49.609791+0000","last_clean":"2024-09-21T22:51:49.609791+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:51:49.609791+0000","last_undegraded":"2024-09-21T22:51:49.609791+0000","last_fullsized":"2024-09-21T22:51:49.609791+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:23.774366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022744280000000002,"stat_sum":{"num_bytes":25266336,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":35,"num_read":7476,"num_read_kb":20350,"num_write":771,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.f","version":"97'404","reported_seq":5051,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.610488+0000","last_change":"2024-09-21T22:51:49.611118+0000","last_active":"2024-09-21T22:51:49.610488+0000","last_peered":"2024-09-21T22:51:49.610488+0000","last_clean":"2024-09-21T22:51:49.610488+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:51:49.610488+0000","last_undegraded":"2024-09-21T22:51:49.610488+0000","last_fullsized":"2024-09-21T22:51:49.610488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:09.219643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028618509999999999,"stat_sum":{"num_bytes":27808362,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":5866,"num_read_kb":8765,"num_write":450,"num_write_kb":2859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.10","version":"100'546","reported_seq":7138,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.672608+0000","last_change":"2024-09-21T22:51:51.516122+0000","last_active":"2024-09-21T22:51:51.672608+0000","last_peered":"2024-09-21T22:51:51.672608+0000","last_clean":"2024-09-21T22:51:51.672608+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:51:51.672608+0000","last_undegraded":"2024-09-21T22:51:51.672608+0000","last_fullsized":"2024-09-21T22:51:51.672608+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:54.355503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039277300000000002,"stat_sum":{"num_bytes":37186745,"num_objects":79,"num_object_clones":26,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":38,"num_read":8406,"num_read_kb":38695,"num_write":565,"num_write_kb":9753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"6"}]},{"pgid":"3.11","version":"98'890","reported_seq":5624,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.712331+0000","last_change":"2024-09-21T22:51:49.608731+0000","last_active":"2024-09-21T22:51:49.712331+0000","last_peered":"2024-09-21T22:51:49.712331+0000","last_clean":"2024-09-21T22:51:49.712331+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:51:49.712331+0000","last_undegraded":"2024-09-21T22:51:49.712331+0000","last_fullsized":"2024-09-21T22:51:49.712331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:02.838083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001485538,"stat_sum":{"num_bytes":33075103,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":32,"num_read":7278,"num_read_kb":52267,"num_write":1228,"num_write_kb":30194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.12","version":"95'471","reported_seq":7984,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494716+0000","last_change":"2024-09-21T22:51:49.495121+0000","last_active":"2024-09-21T22:51:49.494716+0000","last_peered":"2024-09-21T22:51:49.494716+0000","last_clean":"2024-09-21T22:51:49.494716+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:51:49.494716+0000","last_undegraded":"2024-09-21T22:51:49.494716+0000","last_fullsized":"2024-09-21T22:51:49.494716+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:50:57.214391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088587099999999999,"stat_sum":{"num_bytes":30083111,"num_objects":82,"num_object_clones":21,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":39,"num_read":9315,"num_read_kb":22667,"num_write":450,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.13","version":"98'524","reported_seq":7499,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.653073+0000","last_change":"2024-09-21T22:51:49.608741+0000","last_active":"2024-09-21T22:51:49.653073+0000","last_peered":"2024-09-21T22:51:49.653073+0000","last_clean":"2024-09-21T22:51:49.653073+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:51:49.653073+0000","last_undegraded":"2024-09-21T22:51:49.653073+0000","last_fullsized":"2024-09-21T22:51:49.653073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:37.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042772,"stat_sum":{"num_bytes":35913549,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":51,"num_read":7382,"num_read_kb":28275,"num_write":609,"num_write_kb":14949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":712783078,"num_objects":2355,"num_object_clones":675,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":1123,"num_read":212852,"num_read_kb":710189,"num_write":22715,"num_write_kb":331537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":452014080,"data_stored":501027845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":18442,"ondisk_log_size":18442,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476763,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96096,"kb_used_data":46848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94275744,"statfs":{"total":96636764160,"available":96538361856,"internally_reserved":0,"allocated":47972352,"data_stored":56060322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115712,"kb_used_data":66472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94256128,"statfs":{"total":96636764160,"available":96518275072,"internally_reserved":0,"allocated":68067328,"data_stored":73473909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42060,"kb_used_data":9784,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94329780,"statfs":{"total":96636764160,"available":96593694720,"internally_reserved":0,"allocated":10018816,"data_stored":12673842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":177400,"kb_used_data":101660,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94194440,"statfs":{"total":96636764160,"available":96455106560,"internally_reserved":0,"allocated":104099840,"data_stored":117323289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51724,"kb_used_data":12156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320116,"statfs":{"total":96636764160,"available":96583798784,"internally_reserved":0,"allocated":12447744,"data_stored":13895880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70660,"kb_used_data":39464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301180,"statfs":{"total":96636764160,"available":96564408320,"internally_reserved":0,"allocated":40411136,"data_stored":42083696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":69624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":71294976,"data_stored":79783189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056643,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134964,"kb_used_data":105628,"kb_used_omap":42,"kb_used_meta":29269,"kb_avail":94236876,"statfs":{"total":96636764160,"available":96498561024,"internally_reserved":0,"allocated":108163072,"data_stored":111192708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":29972469},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":78659602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":39251968,"data_stored":41550458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11304960,"data_stored":13368893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":107008000,"data_stored":110659489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102944768,"data_stored":116790070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46800896,"data_stored":55520852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8839168,"data_stored":12128159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":66318336,"data_stored":72350322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:51:52.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:51:52.984 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:52.979+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:53.014 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.014 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:53.117 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:53.115+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:53.185 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:53.184+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:51:53.287 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:53.284+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:53.291 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-21T22:51:52.509618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":713373465,"num_objects":2359,"num_object_clones":675,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":1123,"num_read":212968,"num_read_kb":710289,"num_write":22876,"num_write_kb":333955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18562,"ondisk_log_size":18562,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":811644,"kb_used_data":451636,"kb_used_omap":55,"kb_used_meta":359560,"kb_avail":754163076,"statfs":{"total":773094113280,"available":772262989824,"internally_reserved":0,"allocated":462475264,"data_stored":506486835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":368189802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-203293705,"num_objects":-86,"num_object_clones":-25,"num_object_copies":-172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-86,"num_whiteouts":-11,"num_read":863,"num_read_kb":733,"num_write":186,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":5834135,"num_keys_recovered":12,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020946"},"pg_stats":[{"pgid":"3.1f","version":"96'921","reported_seq":6926,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497725+0000","last_change":"2024-09-21T22:51:49.498031+0000","last_active":"2024-09-21T22:51:49.497725+0000","last_peered":"2024-09-21T22:51:49.497725+0000","last_clean":"2024-09-21T22:51:49.497725+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:51:49.497725+0000","last_undegraded":"2024-09-21T22:51:49.497725+0000","last_fullsized":"2024-09-21T22:51:49.497725+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:18:38.222143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014314040000000001,"stat_sum":{"num_bytes":31556376,"num_objects":83,"num_object_clones":26,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":35,"num_read":9488,"num_read_kb":21669,"num_write":1212,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1e","version":"95'479","reported_seq":5818,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497401+0000","last_change":"2024-09-21T22:51:49.497672+0000","last_active":"2024-09-21T22:51:49.497401+0000","last_peered":"2024-09-21T22:51:49.497401+0000","last_clean":"2024-09-21T22:51:49.497401+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:51:49.497401+0000","last_undegraded":"2024-09-21T22:51:49.497401+0000","last_fullsized":"2024-09-21T22:51:49.497401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:04:28.748600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272022,"stat_sum":{"num_bytes":25291019,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":43,"num_read":7105,"num_read_kb":20265,"num_write":451,"num_write_kb":5427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1d","version":"99'970","reported_seq":6999,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.522498+0000","last_change":"2024-09-21T22:51:50.522787+0000","last_active":"2024-09-21T22:51:50.522498+0000","last_peered":"2024-09-21T22:51:50.522498+0000","last_clean":"2024-09-21T22:51:50.522498+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:51:50.522498+0000","last_undegraded":"2024-09-21T22:51:50.522498+0000","last_fullsized":"2024-09-21T22:51:50.522498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:56.386675+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013660189999999999,"stat_sum":{"num_bytes":29531025,"num_objects":77,"num_object_clones":18,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":37,"num_read":10236,"num_read_kb":47062,"num_write":1452,"num_write_kb":18437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.1c","version":"99'756","reported_seq":1983,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.562402+0000","last_change":"2024-09-21T22:51:50.509123+0000","last_active":"2024-09-21T22:51:50.562402+0000","last_peered":"2024-09-21T22:51:50.562402+0000","last_clean":"2024-09-21T22:51:50.562402+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:51:50.562402+0000","last_undegraded":"2024-09-21T22:51:50.562402+0000","last_fullsized":"2024-09-21T22:51:50.562402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:20.773179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013178599999999999,"stat_sum":{"num_bytes":8102632,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":37,"num_read":3471,"num_read_kb":10834,"num_write":1023,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.1b","version":"96'1007","reported_seq":6290,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495592+0000","last_change":"2024-09-21T22:51:49.496285+0000","last_active":"2024-09-21T22:51:49.495592+0000","last_peered":"2024-09-21T22:51:49.495592+0000","last_clean":"2024-09-21T22:51:49.495592+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:51:49.495592+0000","last_undegraded":"2024-09-21T22:51:49.495592+0000","last_fullsized":"2024-09-21T22:51:49.495592+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:39.587519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011844990000000001,"stat_sum":{"num_bytes":29086151,"num_objects":71,"num_object_clones":25,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":9424,"num_read_kb":37686,"num_write":1574,"num_write_kb":12277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1a","version":"98'735","reported_seq":8260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.708618+0000","last_change":"2024-09-21T22:51:49.496765+0000","last_active":"2024-09-21T22:51:49.708618+0000","last_peered":"2024-09-21T22:51:49.708618+0000","last_clean":"2024-09-21T22:51:49.708618+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:51:49.708618+0000","last_undegraded":"2024-09-21T22:51:49.708618+0000","last_fullsized":"2024-09-21T22:51:49.708618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:42:35.320223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010989179999999999,"stat_sum":{"num_bytes":13332042,"num_objects":73,"num_object_clones":25,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":9793,"num_read_kb":12962,"num_write":1085,"num_write_kb":4809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.19","version":"98'474","reported_seq":8347,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651773+0000","last_change":"2024-09-21T22:51:49.651773+0000","last_active":"2024-09-21T22:51:49.651773+0000","last_peered":"2024-09-21T22:51:49.651773+0000","last_clean":"2024-09-21T22:51:49.651773+0000","last_became_active":"2024-09-21T22:51:42.461781+0000","last_became_peered":"2024-09-21T22:51:42.461781+0000","last_unstale":"2024-09-21T22:51:49.651773+0000","last_undegraded":"2024-09-21T22:51:49.651773+0000","last_fullsized":"2024-09-21T22:51:49.651773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:35:10.224125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013902039999999999,"stat_sum":{"num_bytes":39383560,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":8664,"num_read_kb":60910,"num_write":491,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":20755516,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.18","version":"87'1019","reported_seq":11056,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494610+0000","last_change":"2024-09-21T22:51:49.496744+0000","last_active":"2024-09-21T22:51:49.494610+0000","last_peered":"2024-09-21T22:51:49.494610+0000","last_clean":"2024-09-21T22:51:49.494610+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:51:49.494610+0000","last_undegraded":"2024-09-21T22:51:49.494610+0000","last_fullsized":"2024-09-21T22:51:49.494610+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:44:22.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025291580000000001,"stat_sum":{"num_bytes":23998665,"num_objects":80,"num_object_clones":23,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":14180,"num_read_kb":27547,"num_write":1639,"num_write_kb":11372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.17","version":"98'414","reported_seq":3030,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.711868+0000","last_change":"2024-09-21T22:51:49.496754+0000","last_active":"2024-09-21T22:51:49.711868+0000","last_peered":"2024-09-21T22:51:49.711868+0000","last_clean":"2024-09-21T22:51:49.711868+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:51:49.711868+0000","last_undegraded":"2024-09-21T22:51:49.711868+0000","last_fullsized":"2024-09-21T22:51:49.711868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:52.369390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015310020000000001,"stat_sum":{"num_bytes":6684916,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":3539,"num_read_kb":6160,"num_write":395,"num_write_kb":3436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.16","version":"93'380","reported_seq":8436,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.522918+0000","last_change":"2024-09-21T22:51:50.523156+0000","last_active":"2024-09-21T22:51:50.522918+0000","last_peered":"2024-09-21T22:51:50.522918+0000","last_clean":"2024-09-21T22:51:50.522918+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:51:50.522918+0000","last_undegraded":"2024-09-21T22:51:50.522918+0000","last_fullsized":"2024-09-21T22:51:50.522918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:08.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012721846,"stat_sum":{"num_bytes":17639789,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":40,"num_read":7939,"num_read_kb":9321,"num_write":332,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.15","version":"95'407","reported_seq":11093,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.608927+0000","last_change":"2024-09-21T22:51:49.609657+0000","last_active":"2024-09-21T22:51:49.608927+0000","last_peered":"2024-09-21T22:51:49.608927+0000","last_clean":"2024-09-21T22:51:49.608927+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:51:49.608927+0000","last_undegraded":"2024-09-21T22:51:49.608927+0000","last_fullsized":"2024-09-21T22:51:49.608927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:48:02.236052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018147949999999999,"stat_sum":{"num_bytes":40696842,"num_objects":79,"num_object_clones":19,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":11080,"num_read_kb":42972,"num_write":427,"num_write_kb":18958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.14","version":"96'429","reported_seq":9165,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.496357+0000","last_change":"2024-09-21T22:51:49.497022+0000","last_active":"2024-09-21T22:51:49.496357+0000","last_peered":"2024-09-21T22:51:49.496357+0000","last_clean":"2024-09-21T22:51:49.496357+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:51:49.496357+0000","last_undegraded":"2024-09-21T22:51:49.496357+0000","last_fullsized":"2024-09-21T22:51:49.496357+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:37:51.248758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011734429999999999,"stat_sum":{"num_bytes":24498538,"num_objects":71,"num_object_clones":20,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":34,"num_read":9237,"num_read_kb":22032,"num_write":505,"num_write_kb":15782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.515676+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:51:51.515676+0000","last_peered":"2024-09-21T22:51:51.515676+0000","last_clean":"2024-09-21T22:51:51.515676+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:51:51.515676+0000","last_undegraded":"2024-09-21T22:51:51.515676+0000","last_fullsized":"2024-09-21T22:51:51.515676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'508","reported_seq":6411,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495741+0000","last_change":"2024-09-21T22:51:49.495741+0000","last_active":"2024-09-21T22:51:49.495741+0000","last_peered":"2024-09-21T22:51:49.495741+0000","last_clean":"2024-09-21T22:51:49.495741+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:51:49.495741+0000","last_undegraded":"2024-09-21T22:51:49.495741+0000","last_fullsized":"2024-09-21T22:51:49.495741+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:57.618096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096816600000000001,"stat_sum":{"num_bytes":18207663,"num_objects":70,"num_object_clones":20,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":5985,"num_read_kb":22287,"num_write":501,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.650382+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:51:49.650382+0000","last_peered":"2024-09-21T22:51:49.650382+0000","last_clean":"2024-09-21T22:51:49.650382+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:51:49.650382+0000","last_undegraded":"2024-09-21T22:51:49.650382+0000","last_fullsized":"2024-09-21T22:51:49.650382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"98'439","reported_seq":4089,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.498036+0000","last_change":"2024-09-21T22:51:49.498036+0000","last_active":"2024-09-21T22:51:49.498036+0000","last_peered":"2024-09-21T22:51:49.498036+0000","last_clean":"2024-09-21T22:51:49.498036+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:51:49.498036+0000","last_undegraded":"2024-09-21T22:51:49.498036+0000","last_fullsized":"2024-09-21T22:51:49.498036+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:50.919666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013130559999999999,"stat_sum":{"num_bytes":7934304,"num_objects":75,"num_object_clones":23,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":38,"num_read":4761,"num_read_kb":6226,"num_write":503,"num_write_kb":3540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.608075+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:51:49.608075+0000","last_peered":"2024-09-21T22:51:49.608075+0000","last_clean":"2024-09-21T22:51:49.608075+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:51:49.608075+0000","last_undegraded":"2024-09-21T22:51:49.608075+0000","last_fullsized":"2024-09-21T22:51:49.608075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"85'538","reported_seq":2461,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651833+0000","last_change":"2024-09-21T22:51:49.652184+0000","last_active":"2024-09-21T22:51:49.651833+0000","last_peered":"2024-09-21T22:51:49.651833+0000","last_clean":"2024-09-21T22:51:49.651833+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:51:49.651833+0000","last_undegraded":"2024-09-21T22:51:49.651833+0000","last_fullsized":"2024-09-21T22:51:49.651833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:29.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017327099999999999,"stat_sum":{"num_bytes":11458158,"num_objects":80,"num_object_clones":27,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":37,"num_read":2835,"num_read_kb":6413,"num_write":661,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.650589+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:51:49.650589+0000","last_peered":"2024-09-21T22:51:49.650589+0000","last_clean":"2024-09-21T22:51:49.650589+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:51:49.650589+0000","last_undegraded":"2024-09-21T22:51:49.650589+0000","last_fullsized":"2024-09-21T22:51:49.650589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"99'535","reported_seq":2499,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.527180+0000","last_change":"2024-09-21T22:51:50.527824+0000","last_active":"2024-09-21T22:51:50.527180+0000","last_peered":"2024-09-21T22:51:50.527180+0000","last_clean":"2024-09-21T22:51:50.527180+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:51:50.527180+0000","last_undegraded":"2024-09-21T22:51:50.527180+0000","last_fullsized":"2024-09-21T22:51:50.527180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:34:22.521298+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018022554,"stat_sum":{"num_bytes":9171086,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":29,"num_read":3525,"num_read_kb":13902,"num_write":533,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.3","version":"98'533","reported_seq":6161,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.679507+0000","last_change":"2024-09-21T22:51:49.499945+0000","last_active":"2024-09-21T22:51:49.679507+0000","last_peered":"2024-09-21T22:51:49.679507+0000","last_clean":"2024-09-21T22:51:49.679507+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:51:49.679507+0000","last_undegraded":"2024-09-21T22:51:49.679507+0000","last_fullsized":"2024-09-21T22:51:49.679507+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:39:09.978813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034278889999999999,"stat_sum":{"num_bytes":16579695,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":7810,"num_read_kb":13271,"num_write":510,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.2","version":"23'2","reported_seq":173,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.511119+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:51:50.511119+0000","last_peered":"2024-09-21T22:51:50.511119+0000","last_clean":"2024-09-21T22:51:50.511119+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:51:50.511119+0000","last_undegraded":"2024-09-21T22:51:50.511119+0000","last_fullsized":"2024-09-21T22:51:50.511119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"85'578","reported_seq":4537,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.651610+0000","last_change":"2024-09-21T22:51:49.651984+0000","last_active":"2024-09-21T22:51:49.651610+0000","last_peered":"2024-09-21T22:51:49.651610+0000","last_clean":"2024-09-21T22:51:49.651610+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:51:49.651610+0000","last_undegraded":"2024-09-21T22:51:49.651610+0000","last_fullsized":"2024-09-21T22:51:49.651610+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:38.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015154140000000001,"stat_sum":{"num_bytes":25917914,"num_objects":84,"num_object_clones":27,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":37,"num_read":5773,"num_read_kb":19270,"num_write":873,"num_write_kb":12098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":176,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.136075+0000","last_change":"2024-09-21T22:51:42.453022+0000","last_active":"2024-09-21T22:51:50.136075+0000","last_peered":"2024-09-21T22:51:50.136075+0000","last_clean":"2024-09-21T22:51:50.136075+0000","last_became_active":"2024-09-21T22:51:42.452732+0000","last_became_peered":"2024-09-21T22:51:42.452732+0000","last_unstale":"2024-09-21T22:51:50.136075+0000","last_undegraded":"2024-09-21T22:51:50.136075+0000","last_fullsized":"2024-09-21T22:51:50.136075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:04.156384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"95'341","reported_seq":5505,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494544+0000","last_change":"2024-09-21T22:51:49.495336+0000","last_active":"2024-09-21T22:51:49.494544+0000","last_peered":"2024-09-21T22:51:49.494544+0000","last_clean":"2024-09-21T22:51:49.494544+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:51:49.494544+0000","last_undegraded":"2024-09-21T22:51:49.494544+0000","last_fullsized":"2024-09-21T22:51:49.494544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:50:01.343913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011115140000000001,"stat_sum":{"num_bytes":38151644,"num_objects":83,"num_object_clones":24,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":37,"num_read":5111,"num_read_kb":47271,"num_write":305,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.606762+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:51:49.606762+0000","last_peered":"2024-09-21T22:51:49.606762+0000","last_clean":"2024-09-21T22:51:49.606762+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:51:49.606762+0000","last_undegraded":"2024-09-21T22:51:49.606762+0000","last_fullsized":"2024-09-21T22:51:49.606762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'638","reported_seq":2806,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497316+0000","last_change":"2024-09-21T22:51:49.497651+0000","last_active":"2024-09-21T22:51:49.497316+0000","last_peered":"2024-09-21T22:51:49.497316+0000","last_clean":"2024-09-21T22:51:49.497316+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:51:49.497316+0000","last_undegraded":"2024-09-21T22:51:49.497316+0000","last_fullsized":"2024-09-21T22:51:49.497316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:43.085622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574935,"stat_sum":{"num_bytes":18088482,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":3750,"num_read_kb":14069,"num_write":812,"num_write_kb":10973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.3","version":"21'1","reported_seq":172,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.507593+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:51:50.507593+0000","last_peered":"2024-09-21T22:51:50.507593+0000","last_clean":"2024-09-21T22:51:50.507593+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:51:50.507593+0000","last_undegraded":"2024-09-21T22:51:50.507593+0000","last_fullsized":"2024-09-21T22:51:50.507593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.607030+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:51:49.607030+0000","last_peered":"2024-09-21T22:51:49.607030+0000","last_clean":"2024-09-21T22:51:49.607030+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:51:49.607030+0000","last_undegraded":"2024-09-21T22:51:49.607030+0000","last_fullsized":"2024-09-21T22:51:49.607030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"97'692","reported_seq":2698,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.495771+0000","last_change":"2024-09-21T22:51:49.496992+0000","last_active":"2024-09-21T22:51:49.495771+0000","last_peered":"2024-09-21T22:51:49.495771+0000","last_clean":"2024-09-21T22:51:49.495771+0000","last_became_active":"2024-09-21T22:51:45.471804+0000","last_became_peered":"2024-09-21T22:51:45.471804+0000","last_unstale":"2024-09-21T22:51:49.495771+0000","last_undegraded":"2024-09-21T22:51:49.495771+0000","last_fullsized":"2024-09-21T22:51:49.495771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:51:25.842675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601253,"stat_sum":{"num_bytes":14493799,"num_objects":73,"num_object_clones":19,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":4140,"num_read_kb":7282,"num_write":936,"num_write_kb":5004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1097517,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.9","version":"97'765","reported_seq":5428,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.497463+0000","last_change":"2024-09-21T22:51:49.498064+0000","last_active":"2024-09-21T22:51:49.497463+0000","last_peered":"2024-09-21T22:51:49.497463+0000","last_clean":"2024-09-21T22:51:49.497463+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:51:49.497463+0000","last_undegraded":"2024-09-21T22:51:49.497463+0000","last_fullsized":"2024-09-21T22:51:49.497463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:34:04.934264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001891053,"stat_sum":{"num_bytes":7674582,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":38,"num_read":6837,"num_read_kb":10243,"num_write":973,"num_write_kb":5643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.a","version":"100'298","reported_seq":1247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.596691+0000","last_change":"2024-09-21T22:51:51.516319+0000","last_active":"2024-09-21T22:51:51.596691+0000","last_peered":"2024-09-21T22:51:51.596691+0000","last_clean":"2024-09-21T22:51:51.596691+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:51:51.596691+0000","last_undegraded":"2024-09-21T22:51:51.596691+0000","last_fullsized":"2024-09-21T22:51:51.596691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:38.103227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057854900000000003,"stat_sum":{"num_bytes":13629912,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":891,"num_read_kb":14115,"num_write":288,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"6"}]},{"pgid":"3.b","version":"85'317","reported_seq":1391,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.496839+0000","last_change":"2024-09-21T22:51:49.497188+0000","last_active":"2024-09-21T22:51:49.496839+0000","last_peered":"2024-09-21T22:51:49.496839+0000","last_clean":"2024-09-21T22:51:49.496839+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:51:49.496839+0000","last_undegraded":"2024-09-21T22:51:49.496839+0000","last_fullsized":"2024-09-21T22:51:49.496839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:49.354546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001208277,"stat_sum":{"num_bytes":24945951,"num_objects":79,"num_object_clones":25,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":36,"num_read":1415,"num_read_kb":18056,"num_write":315,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.c","version":"99'290","reported_seq":5163,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-21T22:51:50.521378+0000","last_change":"2024-09-21T22:51:50.521810+0000","last_active":"2024-09-21T22:51:50.521378+0000","last_peered":"2024-09-21T22:51:50.521378+0000","last_clean":"2024-09-21T22:51:50.521378+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:51:50.521378+0000","last_undegraded":"2024-09-21T22:51:50.521378+0000","last_fullsized":"2024-09-21T22:51:50.521378+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:02:35.479805+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013250288000000001,"stat_sum":{"num_bytes":19503041,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":4881,"num_read_kb":18947,"num_write":309,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.d","version":"97'569","reported_seq":3300,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.609077+0000","last_change":"2024-09-21T22:51:49.610360+0000","last_active":"2024-09-21T22:51:49.609077+0000","last_peered":"2024-09-21T22:51:49.609077+0000","last_clean":"2024-09-21T22:51:49.609077+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:51:49.609077+0000","last_undegraded":"2024-09-21T22:51:49.609077+0000","last_fullsized":"2024-09-21T22:51:49.609077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:41.619652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024746120000000002,"stat_sum":{"num_bytes":7892086,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":36,"num_read":5259,"num_read_kb":8398,"num_write":537,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.e","version":"96'575","reported_seq":6196,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.609791+0000","last_change":"2024-09-21T22:51:49.610374+0000","last_active":"2024-09-21T22:51:49.609791+0000","last_peered":"2024-09-21T22:51:49.609791+0000","last_clean":"2024-09-21T22:51:49.609791+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:51:49.609791+0000","last_undegraded":"2024-09-21T22:51:49.609791+0000","last_fullsized":"2024-09-21T22:51:49.609791+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:23.774366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022744280000000002,"stat_sum":{"num_bytes":25266336,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":35,"num_read":7476,"num_read_kb":20350,"num_write":771,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.f","version":"97'404","reported_seq":5051,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.610488+0000","last_change":"2024-09-21T22:51:49.611118+0000","last_active":"2024-09-21T22:51:49.610488+0000","last_peered":"2024-09-21T22:51:49.610488+0000","last_clean":"2024-09-21T22:51:49.610488+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:51:49.610488+0000","last_undegraded":"2024-09-21T22:51:49.610488+0000","last_fullsized":"2024-09-21T22:51:49.610488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:09.219643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028618509999999999,"stat_sum":{"num_bytes":27808362,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":5866,"num_read_kb":8765,"num_write":450,"num_write_kb":2859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.10","version":"100'546","reported_seq":7138,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T22:51:51.672608+0000","last_change":"2024-09-21T22:51:51.516122+0000","last_active":"2024-09-21T22:51:51.672608+0000","last_peered":"2024-09-21T22:51:51.672608+0000","last_clean":"2024-09-21T22:51:51.672608+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:51:51.672608+0000","last_undegraded":"2024-09-21T22:51:51.672608+0000","last_fullsized":"2024-09-21T22:51:51.672608+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:54.355503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039277300000000002,"stat_sum":{"num_bytes":37186745,"num_objects":79,"num_object_clones":26,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":38,"num_read":8406,"num_read_kb":38695,"num_write":565,"num_write_kb":9753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"6"}]},{"pgid":"3.11","version":"98'890","reported_seq":5624,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.712331+0000","last_change":"2024-09-21T22:51:49.608731+0000","last_active":"2024-09-21T22:51:49.712331+0000","last_peered":"2024-09-21T22:51:49.712331+0000","last_clean":"2024-09-21T22:51:49.712331+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:51:49.712331+0000","last_undegraded":"2024-09-21T22:51:49.712331+0000","last_fullsized":"2024-09-21T22:51:49.712331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:02.838083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001485538,"stat_sum":{"num_bytes":33075103,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":32,"num_read":7278,"num_read_kb":52267,"num_write":1228,"num_write_kb":30194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.12","version":"95'471","reported_seq":7984,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.494716+0000","last_change":"2024-09-21T22:51:49.495121+0000","last_active":"2024-09-21T22:51:49.494716+0000","last_peered":"2024-09-21T22:51:49.494716+0000","last_clean":"2024-09-21T22:51:49.494716+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:51:49.494716+0000","last_undegraded":"2024-09-21T22:51:49.494716+0000","last_fullsized":"2024-09-21T22:51:49.494716+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:50:57.214391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088587099999999999,"stat_sum":{"num_bytes":30083111,"num_objects":82,"num_object_clones":21,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":39,"num_read":9315,"num_read_kb":22667,"num_write":450,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.13","version":"98'524","reported_seq":7499,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T22:51:49.653073+0000","last_change":"2024-09-21T22:51:49.608741+0000","last_active":"2024-09-21T22:51:49.653073+0000","last_peered":"2024-09-21T22:51:49.653073+0000","last_clean":"2024-09-21T22:51:49.653073+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:51:49.653073+0000","last_undegraded":"2024-09-21T22:51:49.653073+0000","last_fullsized":"2024-09-21T22:51:49.653073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:37.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042772,"stat_sum":{"num_bytes":35913549,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":51,"num_read":7382,"num_read_kb":28275,"num_write":609,"num_write_kb":14949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":712783078,"num_objects":2355,"num_object_clones":675,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":1123,"num_read":212852,"num_read_kb":710189,"num_write":22715,"num_write_kb":331537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":452014080,"data_stored":501027845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":18442,"ondisk_log_size":18442,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476763,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96096,"kb_used_data":46848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94275744,"statfs":{"total":96636764160,"available":96538361856,"internally_reserved":0,"allocated":47972352,"data_stored":56060322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115712,"kb_used_data":66472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94256128,"statfs":{"total":96636764160,"available":96518275072,"internally_reserved":0,"allocated":68067328,"data_stored":73473909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42060,"kb_used_data":9784,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94329780,"statfs":{"total":96636764160,"available":96593694720,"internally_reserved":0,"allocated":10018816,"data_stored":12673842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":177400,"kb_used_data":101660,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94194440,"statfs":{"total":96636764160,"available":96455106560,"internally_reserved":0,"allocated":104099840,"data_stored":117323289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51724,"kb_used_data":12156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320116,"statfs":{"total":96636764160,"available":96583798784,"internally_reserved":0,"allocated":12447744,"data_stored":13895880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70660,"kb_used_data":39464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301180,"statfs":{"total":96636764160,"available":96564408320,"internally_reserved":0,"allocated":40411136,"data_stored":42083696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":69624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":71294976,"data_stored":79783189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056643,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134964,"kb_used_data":105628,"kb_used_omap":42,"kb_used_meta":29269,"kb_avail":94236876,"statfs":{"total":96636764160,"available":96498561024,"internally_reserved":0,"allocated":108163072,"data_stored":111192708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":29972469},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":78659602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":39251968,"data_stored":41550458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11304960,"data_stored":13368893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":107008000,"data_stored":110659489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102944768,"data_stored":116790070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46800896,"data_stored":55520852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8839168,"data_stored":12128159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":66318336,"data_stored":72350322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:51:53.293 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T22:51:53.293 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:51:53.293 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T22:51:53.293 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T22:51:53.293 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T22:51:53.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T22:51:53.373 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.373 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.387 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:53.384+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:53.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:53.453 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "" 2024-09-21T22:51:53.454 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:53.487 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:53.488+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:53.506 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.506 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:53.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:51:53.612 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:53.612+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:53.690 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:53.687+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:53.696 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.697 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:53.791 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:53.787+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:53.869 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:53.870 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:53.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:53.892 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:53.887+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:53.993 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:53.987+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:53.999 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.000 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:54.094 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:54.092+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:54.135 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.135 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:54.194 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:54.192+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:54.295 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:54.291+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:54.312 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.312 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:54.396 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:54.395+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:54.447 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.447 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:54.497 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:54.495+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:54.520 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:51:54.594 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.595 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.598 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:54.595+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:54.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:54.699 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:54.695+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:54.737 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.737 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:54.799 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:54.796+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:54.873 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:54.873 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:54.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:54.932 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:54.932+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:54.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:55.001 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:54.999+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:55.054 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:55.055 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:55.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:55.102 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:55.099+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:55.115 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:55.116 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:55.132 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:55.149 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:55.149 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:55.166 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:55.186 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:55.186 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:3791 3.0 e32/24 3:06360417:::rbd_data.111e7572a6e.0000000000000001:head v 32'40, mlcod=32'38)", 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 96.964606434000004, 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038229190000000001, 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227227+0000", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227229+0000", 2024-09-21T22:51:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227734+0000", 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050512400000000002 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227766+0000", 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2449000000000001e-05 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227780+0000", 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3509000000000001e-05 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227911+0000", 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131352 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.227964+0000", 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2911999999999998e-05 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.230982+0000", 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003017877 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:18.231050+0000", 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8096000000000007e-05 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:21148 3.9 e49/24 3:90a2c379:::rbd_data.111e38708da6.0000000000000148:head v 49'109, mlcod=48'107)", 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.148043435999995, 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036214749999999999, 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043790+0000", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043791+0000", 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2720000000000001e-06 2024-09-21T22:51:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043805+0000", 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3711e-05 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043819+0000", 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4375999999999999e-05 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043826+0000", 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8369999999999999e-06 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043935+0000", 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010859899999999999 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.043970+0000", 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4798999999999997e-05 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047302+0000", 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033321570000000001 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:42.047412+0000", 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109724 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:36629 3.9 e53/24 3:96baffb3:::rbd_data.111e4aac08c7.0000000000000040:head v 53'156, mlcod=51'151)", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 68.971078297999995, 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003438235, 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220755+0000", 2024-09-21T22:51:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220756+0000", 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220764+0000", 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6140000000000003e-06 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220776+0000", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.239e-05 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220788+0000", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1922999999999999e-05 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.220979+0000", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019101 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.221006+0000", 2024-09-21T22:51:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6197e-05 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224111+0000", 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031050380000000001 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.224193+0000", 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2843999999999995e-05 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.359969641999996, 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49505 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'416, mlcod=55'414)", 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.180164442999995, 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031957309999999998, 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011669+0000", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011672+0000", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1099999999999999e-06 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011856+0000", 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018379299999999999 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011904+0000", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000002e-05 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.011929+0000", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119000000000001e-05 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012095+0000", 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000166525 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.012148+0000", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2960000000000001e-05 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014767+0000", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026188449999999999 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.014865+0000", 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7589000000000004e-05 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.102544772000002, 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49512 3.0 e55/24 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head v 55'417, mlcod=55'415)", 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.102279500999998, 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043546360000000003, 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089554+0000", 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089556+0000", 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3470000000000001e-06 2024-09-21T22:51:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089573+0000", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6303e-05 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089604+0000", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1420000000000001e-05 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089623+0000", 2024-09-21T22:51:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9266000000000002e-05 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090442+0000", 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081872800000000001 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.090659+0000", 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217128 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093832+0000", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031725450000000001 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093909+0000", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6898999999999997e-05 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.772206703999998, 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53960 3.9 e59/24 3:92d8c30e:::rbd_data.111e5a344aa.0000000000000027:head v 59'167, mlcod=56'164)", 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.591120754000002, 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036382250000000001, 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600713+0000", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600714+0000", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.009e-06 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600726+0000", 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2e-05 2024-09-21T22:51:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600768+0000", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1831e-05 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.600789+0000", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1542000000000001e-05 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601525+0000", 2024-09-21T22:51:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073561000000000004 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.601578+0000", 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3310000000000003e-05 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604221+0000", 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026428810000000001 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.604351+0000", 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000130042 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.576505388000001, 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53974 3.0 e59/24 3:0421eabd:::rbd_data.111e5a344aa.0000000000000112:head v 59'430, mlcod=55'427)", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.576122796999996, 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035431260000000002, 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615711+0000", 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615712+0000", 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999999e-07 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615721+0000", 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1880000000000008e-06 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615741+0000", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0397000000000001e-05 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615765+0000", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3354e-05 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615940+0000", 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000175453 2024-09-21T22:51:55.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615984+0000", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3819999999999997e-05 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619146+0000", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003161783 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619254+0000", 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010823900000000001 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53982 3.4 e59/24 3:21f1ce10:::rbd_data.111e5a344aa.000000000000011a:head v 59'351, mlcod=56'346)", 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.574110333, 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003430383, 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617723+0000", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617726+0000", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.43e-06 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617745+0000", 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9296999999999998e-05 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617774+0000", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9235999999999999e-05 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617788+0000", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3934000000000001e-05 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617941+0000", 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015259200000000001 2024-09-21T22:51:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.617999+0000", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8062000000000001e-05 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621048+0000", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030496529999999998 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.621154+0000", 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105179 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.545385533000001, 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:51:55.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54013 3.4 e59/24 3:20d136b6:::rbd_data.111e5a344aa.0000000000000020:head v 59'355, mlcod=59'349)", 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.542692762999998, 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031653549999999999, 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649141+0000", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649142+0000", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.562e-06 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649155+0000", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2785999999999999e-05 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649183+0000", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7804999999999999e-05 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649197+0000", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4618e-05 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649290+0000", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2948999999999994e-05 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.649326+0000", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5852000000000002e-05 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652223+0000", 2024-09-21T22:51:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028967060000000002 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.652306+0000", 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3077000000000004e-05 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:147259 3.9 e79/24 3:9126dc93:::rbd_data.111e2790987.0000000000000003:head v 79'256, mlcod=77'254)", 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:55.253 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.782233138999999, 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033560030000000002, 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409600+0000", 2024-09-21T22:51:55.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.409601+0000", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0800000000000004e-07 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410967+0000", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013654699999999999 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410983+0000", 2024-09-21T22:51:55.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999e-05 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.410993+0000", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0431999999999999e-05 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411062+0000", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9331000000000004e-05 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.411084+0000", 2024-09-21T22:51:55.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1390999999999999e-05 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412906+0000", 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001821956 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.412956+0000", 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0615999999999998e-05 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.257 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.095311749, 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168816 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'545, mlcod=87'543)", 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.300568918, 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033834189999999999, 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891265+0000", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891267+0000", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2500000000000001e-06 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891281+0000", 2024-09-21T22:51:55.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3723000000000001e-05 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891323+0000", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2926000000000002e-05 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891344+0000", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0250000000000001e-05 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891475+0000", 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013148000000000001 2024-09-21T22:51:55.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.891506+0000", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0784999999999997e-05 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894584+0000", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003077598 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.894648+0000", 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4406999999999995e-05 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.265 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168818 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'547, mlcod=87'543)", 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.298608561999998, 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039584049999999999, 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893225+0000", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893227+0000", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7460000000000001e-06 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893242+0000", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5067e-05 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893273+0000", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0920000000000002e-05 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893292+0000", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9732999999999999e-05 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893858+0000", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000565209 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.893891+0000", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3822999999999998e-05 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897073+0000", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031815010000000002 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.897183+0000", 2024-09-21T22:51:55.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110406 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:168827 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'552, mlcod=87'549)", 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.291137419000002, 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003413499, 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900696+0000", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900698+0000", 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.787e-06 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900712+0000", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4486e-05 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900746+0000", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3330000000000001e-05 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900765+0000", 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.906e-05 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900904+0000", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139519 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.900980+0000", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5636000000000006e-05 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904045+0000", 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030655510000000001 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:36.904110+0000", 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4129999999999998e-05 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:169241 3.9 e87/24 3:9537a022:::rbd_object_map.111ee98c1da3:head v 87'653, mlcod=87'650)", 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.163795922999999, 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032275400000000001, 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.274 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028038+0000", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028047+0000", 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7059999999999999e-06 2024-09-21T22:51:55.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028061+0000", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.411e-05 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028094+0000", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2292999999999997e-05 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028111+0000", 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.774e-05 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028274+0000", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162442 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.028329+0000", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5112999999999999e-05 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031115+0000", 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002785624 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:55.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:38.031265+0000", 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150512 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:55.278 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:55.279 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:55.279 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:55.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:55.280 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:55.280 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:55.281 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:55.281 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:55.282 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:55.275+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:55.380 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:55.380+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:55.412 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:55.412 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:55.412 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:55.413 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:55.426 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:55.481 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:55.479+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:55.550 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:55.550 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:55.550 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:55.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:51:55.582 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:55.580+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:55.683 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:55.679+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766445771999997, 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:51:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:51:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:51:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766415770999998, 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:51:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:51:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:51:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766388065000001, 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:51:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766342115, 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766301979999994, 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766271981000003, 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:51:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:51:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766237919999995, 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:51:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:51:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766194064999993, 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:51:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766157934000006, 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:51:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766107352999995, 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:51:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:51:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:51:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.766002138999994, 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765938629000004, 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765840427000001, 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765775164000004, 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:51:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:51:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765709247000004, 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:51:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:51:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:51:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765632144999998, 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:51:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:51:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:51:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765558545000005, 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:51:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:51:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:51:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.880457620999998, 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.601597044999998, 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:51:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:51:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:51:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.515838045999999, 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:51:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:55.834 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:55.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:51:55.835 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:55.832+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:55.936 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:55.936+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.010 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:56.010 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:51:56.037 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:56.036+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:56.137 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:56.136+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:56.161 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.161 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:56.161 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:56.162 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:56.162 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:51:56.238 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:56.236+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029308000999997, 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:51:56.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:51:56.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029266155000002, 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:51:56.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:51:56.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:51:56.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029220680999998, 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:51:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:51:56.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029185753, 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:51:56.326 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:51:56.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029153506999997, 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:51:56.331 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:51:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:51:56.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:51:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029117775000003, 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029080598999997, 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.029046706000003, 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:51:56.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.020611361999997, 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351594629000001, 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:51:56.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.35154301, 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351491707000001, 2024-09-21T22:51:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:51:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:51:56.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351433791000002, 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:51:56.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:51:56.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:51:56.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351378132000001, 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:51:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:51:56.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:56.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351321982999998, 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351259491, 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:51:56.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351158673, 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:51:56.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.391 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351102854000001, 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:51:56.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.351050147999999, 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:51:56.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.802956475, 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:51:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.407 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.407 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.407 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:51:56.409 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:56.408+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:56.509 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:56.503+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:56.536 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.536 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:56.536 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:56.537 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:51:56.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:51:56.637 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:56.635+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:56.711 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:56.707+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:56.718 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:56.718 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:51:56.817 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:56.815+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:51:56.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 123.962211035, 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:56.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:51:56.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:51:56.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.417139176000006, 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:51:56.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:51:56.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:51:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:51:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.417075725999993, 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.417012548000002, 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416956208000002, 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.922 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416914465999994, 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:56.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416853864999993, 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:51:56.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416773429000003, 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:51:56.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.932 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:56.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.936 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416697318999994, 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:51:56.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416610246000005, 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:51:56.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416566758000002, 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:56.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:51:56.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:51:56.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416533119999997, 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:51:56.949 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:51:56.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416499353999995, 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:51:56.953 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:51:56.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:51:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.957 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416468069999993, 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416422714999996, 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416391219000005, 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:51:56.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:51:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:51:56.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:51:56.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.416356437999994, 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.398349863000007, 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:51:56.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:51:56.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.075769713, 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:51:56.978 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:51:56.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:51:56.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.374331524, 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:56.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:56.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:56.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:56.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:56.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:56.990 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:56.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:51:56.991 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:56.988+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:57.044 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:51:57.092 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:57.087+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:57.168 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:57.168 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:57.183 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:51:57.225 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:57.223+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:57.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:51:57.328 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:57.328+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:57.350 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:57.350 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:57.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:51:57.409 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:57.407+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:57.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:57.439 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:57.455 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:57.456 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:57.472 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:57.472 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:57.487 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:51:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.466444984000006, 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:51:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:51:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:51:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.462207414999995, 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:51:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:51:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:51:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.462012517000005, 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:51:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:51:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740337887999999, 2024-09-21T22:51:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740296008000001, 2024-09-21T22:51:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:51:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:51:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740260421999999, 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:51:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:51:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:51:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.74022824, 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:51:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740196214000001, 2024-09-21T22:51:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740126201999999, 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:51:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.740062322999997, 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:51:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.74000908, 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739957771, 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:51:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739906945999998, 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739859787, 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:51:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:51:57.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739824251000002, 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739776022000001, 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:51:57.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:51:57.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739707879999997, 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:51:57.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:51:57.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739646743999998, 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739588576000003, 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.739532427, 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:51:57.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:51:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:51:57.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:51:57.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:57.578 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:57.579 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:57.579 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:57.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:51:57.580 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:57.575+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:57.581 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:57.581 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:57.582 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:57.680 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:57.680+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:57.745 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:57.745 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:57.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:51:57.780 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:57.779+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:57.881 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:51:57.880+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:51:57.915 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:57.915 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:57.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:51:57.982 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:57.979+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:58.083 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:51:58.079+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:51:58.104 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.104 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:51:58.184 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:58.184+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:58.241 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.241 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:51:58.285 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:58.279+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:58.386 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:51:58.383+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:51:58.424 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.424 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:51:58.468 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 7, 6, 3] 2024-09-21T22:51:58.468 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:51:58.468 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T22:51:58.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T22:51:58.528 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:58.528+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:58.560 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.560 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.575 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:51:58.616 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:58.616+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:58.689 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:58.688+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:58.730 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.730 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:51:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 107, 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "56013142-0739-4aee-ba29-30eddcc2dbe0", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-21T22:49:36.309693+0000", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-21T22:51:58.036233+0000", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-21T22:51:41.430550+0000", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-21T22:49:39.319546+0000", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5806080, 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "nodeep-scrub", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "noscrub", 2024-09-21T22:51:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 7, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-21T22:51:58.779 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:51.363777+0000", 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:51:58.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:51:58.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "19", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:51:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:51:58.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.784 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.785 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:55.226638+0000", 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:51:58.786 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "23", 2024-09-21T22:51:58.787 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 23, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:51:58.788 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:51:58.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:51:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:50:05.084190+0000", 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:51:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-21T22:51:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "107", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 37, 2024-09-21T22:51:58.793 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 107, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:51:58.794 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:51:58.795 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ecfcea04-7198-4c57-bc74-45aaf791e3b7", 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 78, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.797 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.798 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.799 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/2211646084", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/2211646084", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/2211646084", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/2211646084", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.801 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b6b749e6-4a66-4d8b-bf50-c9634a6927d5", 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 92, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.802 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1102146551", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1102146551", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1102146551", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1102146551", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "4e90da65-2fdd-4c85-a207-70d5f22c95a4", 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 90, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-21T22:51:58.808 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/2482161893", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/2482161893", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/2482161893", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/2482161893", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.811 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "3c97651a-402c-4171-9648-c615dd754c94", 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 13, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 77, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 90, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 93, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 78, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-21T22:51:58.812 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/2575188997", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/2575188997", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/2575188997", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/2575188997", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:51:58.816 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b935b00e-0d91-4f2b-9f2d-2c2be1144ca8", 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 24, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.817 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6819", 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6821", 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6823", 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6826", 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6832", 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6833", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6828", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6830", 2024-09-21T22:51:58.820 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6821/1105568802", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6826/1105568802", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6833/1105568802", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6830/1105568802", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "559dea61-5abd-423e-9dfb-cf92f9e6d4a4", 2024-09-21T22:51:58.821 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 16, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 93, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6818", 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6820", 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6822", 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6824", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6829", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6831", 2024-09-21T22:51:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6825", 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6827", 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6820/2691947692", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6824/2691947692", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2691947692", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6827/2691947692", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "aeee922d-53c2-41f4-9433-8a9691f3e674", 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.826 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 15, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 23, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 42, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 78, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 24, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6810", 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6811", 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6812", 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6813", 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6816", 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6817", 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6814", 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6815", 2024-09-21T22:51:58.830 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6811/3470582226", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6813/3470582226", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6817/3470582226", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6815/3470582226", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b40a3234-6950-4a73-9a5c-f0c086450816", 2024-09-21T22:51:58.831 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 90, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6800", 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6801", 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6802", 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.833 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6803", 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6806", 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6807", 2024-09-21T22:51:58.834 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6804", 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6805", 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6801/1713641021", 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6803/1713641021", 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1713641021", 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1713641021", 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.836 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.846676+0000", 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.837 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:47.117051+0000", 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.836357+0000", 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.838 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.351920+0000", 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 78 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.839 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.975489+0000", 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.838876+0000", 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.840 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.842751+0000", 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 24 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:51:58.841 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.784805+0000", 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [ 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.1f", 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: 3, 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: 5 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [], 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-21T22:51:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:51:58.843 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 25, 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 9 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 25, 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.844 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:51:58.845 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:51:58.847 INFO:tasks.thrashosds.thrasher:j is {'epoch': 107, 'fsid': '56013142-0739-4aee-ba29-30eddcc2dbe0', 'created': '2024-09-21T22:49:36.309693+0000', 'modified': '2024-09-21T22:51:58.036233+0000', 'last_up_change': '2024-09-21T22:51:41.430550+0000', 'last_in_change': '2024-09-21T22:49:39.319546+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T22:49:51.363777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T22:49:55.226638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T22:50:05.084190+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ecfcea04-7198-4c57-bc74-45aaf791e3b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 78, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 2211646084}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 2211646084}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 2211646084}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 2211646084}]}, 'public_addr': '172.21.15.94:6809/2211646084', 'cluster_addr': '172.21.15.94:6811/2211646084', 'heartbeat_back_addr': '172.21.15.94:6815/2211646084', 'heartbeat_front_addr': '172.21.15.94:6813/2211646084', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b6b749e6-4a66-4d8b-bf50-c9634a6927d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1102146551}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1102146551}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1102146551}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1102146551}]}, 'public_addr': '172.21.15.94:6825/1102146551', 'cluster_addr': '172.21.15.94:6827/1102146551', 'heartbeat_back_addr': '172.21.15.94:6831/1102146551', 'heartbeat_front_addr': '172.21.15.94:6829/1102146551', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4e90da65-2fdd-4c85-a207-70d5f22c95a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 2482161893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 2482161893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 2482161893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 2482161893}]}, 'public_addr': '172.21.15.94:6817/2482161893', 'cluster_addr': '172.21.15.94:6819/2482161893', 'heartbeat_back_addr': '172.21.15.94:6823/2482161893', 'heartbeat_front_addr': '172.21.15.94:6821/2482161893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3c97651a-402c-4171-9648-c615dd754c94', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 77, 'up_from': 90, 'up_thru': 93, 'down_at': 78, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 2575188997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 2575188997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 2575188997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 2575188997}]}, 'public_addr': '172.21.15.94:6801/2575188997', 'cluster_addr': '172.21.15.94:6803/2575188997', 'heartbeat_back_addr': '172.21.15.94:6807/2575188997', 'heartbeat_front_addr': '172.21.15.94:6805/2575188997', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b935b00e-0d91-4f2b-9f2d-2c2be1144ca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6819', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 1105568802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6823', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6826', 'nonce': 1105568802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 1105568802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6830', 'nonce': 1105568802}]}, 'public_addr': '172.21.15.99:6821/1105568802', 'cluster_addr': '172.21.15.99:6826/1105568802', 'heartbeat_back_addr': '172.21.15.99:6833/1105568802', 'heartbeat_front_addr': '172.21.15.99:6830/1105568802', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '559dea61-5abd-423e-9dfb-cf92f9e6d4a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6820', 'nonce': 2691947692}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6824', 'nonce': 2691947692}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6829', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2691947692}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6825', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2691947692}]}, 'public_addr': '172.21.15.99:6820/2691947692', 'cluster_addr': '172.21.15.99:6824/2691947692', 'heartbeat_back_addr': '172.21.15.99:6831/2691947692', 'heartbeat_front_addr': '172.21.15.99:6827/2691947692', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'aeee922d-53c2-41f4-9433-8a9691f3e674', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 23, 'up_from': 42, 'up_thru': 78, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3470582226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 3470582226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3470582226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3470582226}]}, 'public_addr': '172.21.15.99:6811/3470582226', 'cluster_addr': '172.21.15.99:6813/3470582226', 'heartbeat_back_addr': '172.21.15.99:6817/3470582226', 'heartbeat_front_addr': '172.21.15.99:6815/3470582226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b40a3234-6950-4a73-9a5c-f0c086450816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1713641021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1713641021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1713641021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1713641021}]}, 'public_addr': '172.21.15.99:6801/1713641021', 'cluster_addr': '172.21.15.99:6803/1713641021', 'heartbeat_back_addr': '172.21.15.99:6807/1713641021', 'heartbeat_front_addr': '172.21.15.99:6805/1713641021', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.846676+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:47.117051+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.836357+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.351920+0000', 'dead_epoch': 78}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.975489+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.838876+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.842751+0000', 'dead_epoch': 24}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.784805+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1f', 'osds': [3, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T22:51:58.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:51:58.848 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:58.843+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:58.947 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:58.947+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:58.969 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:58.969 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:58.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:51:59.048 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:59.048+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:59.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:51:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:59.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:51:59.168 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:59.168+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:51:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:51:59.248 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-21T22:51:59.049545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510026344,"num_objects":2171,"num_object_clones":624,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":1079,"num_read":214408,"num_read_kb":711524,"num_write":23148,"num_write_kb":334014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18966,"ondisk_log_size":18966,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":607420,"kb_used_data":243452,"kb_used_omap":55,"kb_used_meta":363720,"kb_avail":754367300,"statfs":{"total":773094113280,"available":772472115200,"internally_reserved":0,"allocated":249294848,"data_stored":279175884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":372449642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-203347121,"num_objects":-188,"num_object_clones":-51,"num_object_copies":-376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-188,"num_whiteouts":-44,"num_read":1440,"num_read_kb":1235,"num_write":272,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.746120"},"pg_stats":[{"pgid":"3.1f","version":"100'923","reported_seq":6973,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.529027+0000","last_change":"2024-09-21T22:51:54.529394+0000","last_active":"2024-09-21T22:51:54.529027+0000","last_peered":"2024-09-21T22:51:54.529027+0000","last_clean":"2024-09-21T22:51:54.529027+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:51:54.529027+0000","last_undegraded":"2024-09-21T22:51:54.529027+0000","last_fullsized":"2024-09-21T22:51:54.529027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":923,"log_dups_size":0,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:23:14.108913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089967120000000005,"stat_sum":{"num_bytes":30507800,"num_objects":81,"num_object_clones":26,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":35,"num_read":9523,"num_read_kb":21699,"num_write":1214,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.1e","version":"100'503","reported_seq":5948,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.529497+0000","last_change":"2024-09-21T22:51:54.529843+0000","last_active":"2024-09-21T22:51:54.529497+0000","last_peered":"2024-09-21T22:51:54.529497+0000","last_clean":"2024-09-21T22:51:54.529497+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:51:54.529497+0000","last_undegraded":"2024-09-21T22:51:54.529497+0000","last_fullsized":"2024-09-21T22:51:54.529497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:21.675020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091417019999999998,"stat_sum":{"num_bytes":24552054,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":42,"num_read":7278,"num_read_kb":20416,"num_write":468,"num_write_kb":5433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.1d","version":"105'1003","reported_seq":7155,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.069583+0000","last_change":"2024-09-21T22:51:56.069583+0000","last_active":"2024-09-21T22:51:56.069583+0000","last_peered":"2024-09-21T22:51:56.069583+0000","last_clean":"2024-09-21T22:51:56.069583+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:51:56.069583+0000","last_undegraded":"2024-09-21T22:51:56.069583+0000","last_fullsized":"2024-09-21T22:51:56.069583+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:07.092153+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021906367,"stat_sum":{"num_bytes":26078097,"num_objects":65,"num_object_clones":15,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":34,"num_read":10438,"num_read_kb":47237,"num_write":1474,"num_write_kb":18443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"7"}]},{"pgid":"3.1c","version":"105'772","reported_seq":2069,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.273318+0000","last_change":"2024-09-21T22:51:56.058594+0000","last_active":"2024-09-21T22:51:56.273318+0000","last_peered":"2024-09-21T22:51:56.273318+0000","last_clean":"2024-09-21T22:51:56.273318+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:51:56.273318+0000","last_undegraded":"2024-09-21T22:51:56.273318+0000","last_fullsized":"2024-09-21T22:51:56.273318+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:44.337122+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.01255307,"stat_sum":{"num_bytes":5171944,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":34,"num_read":3512,"num_read_kb":10875,"num_write":1039,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"7"}]},{"pgid":"3.1b","version":"102'1018","reported_seq":6335,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.526210+0000","last_change":"2024-09-21T22:51:54.527361+0000","last_active":"2024-09-21T22:51:54.526210+0000","last_peered":"2024-09-21T22:51:54.526210+0000","last_clean":"2024-09-21T22:51:54.526210+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:51:54.526210+0000","last_undegraded":"2024-09-21T22:51:54.526210+0000","last_fullsized":"2024-09-21T22:51:54.526210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1018,"log_dups_size":0,"ondisk_log_size":1018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:18:48.860017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031977210000000002,"stat_sum":{"num_bytes":6771143,"num_objects":62,"num_object_clones":22,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":29,"num_read":9424,"num_read_kb":37686,"num_write":1579,"num_write_kb":12277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.1a","version":"100'746","reported_seq":8345,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.696493+0000","last_change":"2024-09-21T22:51:54.523243+0000","last_active":"2024-09-21T22:51:54.696493+0000","last_peered":"2024-09-21T22:51:54.696493+0000","last_clean":"2024-09-21T22:51:54.696493+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:51:54.696493+0000","last_undegraded":"2024-09-21T22:51:54.696493+0000","last_fullsized":"2024-09-21T22:51:54.696493+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:20:19.629629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013596089999999999,"stat_sum":{"num_bytes":9569854,"num_objects":69,"num_object_clones":24,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":9842,"num_read_kb":13008,"num_write":1094,"num_write_kb":4809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.19","version":"101'475","reported_seq":8383,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.653932+0000","last_change":"2024-09-21T22:51:54.654176+0000","last_active":"2024-09-21T22:51:54.653932+0000","last_peered":"2024-09-21T22:51:54.653932+0000","last_clean":"2024-09-21T22:51:54.653932+0000","last_became_active":"2024-09-21T22:51:42.461781+0000","last_became_peered":"2024-09-21T22:51:42.461781+0000","last_unstale":"2024-09-21T22:51:54.653932+0000","last_undegraded":"2024-09-21T22:51:54.653932+0000","last_fullsized":"2024-09-21T22:51:54.653932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:53.856080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048897899999999995,"stat_sum":{"num_bytes":11786301,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":34,"num_read":8664,"num_read_kb":60910,"num_write":492,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":20755516,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.18","version":"103'1026","reported_seq":11098,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.528120+0000","last_change":"2024-09-21T22:51:54.528487+0000","last_active":"2024-09-21T22:51:54.528120+0000","last_peered":"2024-09-21T22:51:54.528120+0000","last_clean":"2024-09-21T22:51:54.528120+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:51:54.528120+0000","last_undegraded":"2024-09-21T22:51:54.528120+0000","last_fullsized":"2024-09-21T22:51:54.528120+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:47.941478+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085870089999999996,"stat_sum":{"num_bytes":20781642,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":14180,"num_read_kb":27547,"num_write":1642,"num_write_kb":11372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.17","version":"101'456","reported_seq":3230,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.524618+0000","last_change":"2024-09-21T22:51:54.525384+0000","last_active":"2024-09-21T22:51:54.524618+0000","last_peered":"2024-09-21T22:51:54.524618+0000","last_clean":"2024-09-21T22:51:54.524618+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:51:54.524618+0000","last_undegraded":"2024-09-21T22:51:54.524618+0000","last_fullsized":"2024-09-21T22:51:54.524618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:36:55.277029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020838160000000001,"stat_sum":{"num_bytes":6160592,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":3821,"num_read_kb":6406,"num_write":425,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.16","version":"105'386","reported_seq":8483,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.065579+0000","last_change":"2024-09-21T22:51:56.066192+0000","last_active":"2024-09-21T22:51:56.065579+0000","last_peered":"2024-09-21T22:51:56.065579+0000","last_clean":"2024-09-21T22:51:56.065579+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:51:56.065579+0000","last_undegraded":"2024-09-21T22:51:56.065579+0000","last_fullsized":"2024-09-21T22:51:56.065579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:24.823825+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.018990366000000002,"stat_sum":{"num_bytes":17168749,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":37,"num_read":7939,"num_read_kb":9321,"num_write":332,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"7"}]},{"pgid":"3.15","version":"103'416","reported_seq":11138,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.545665+0000","last_change":"2024-09-21T22:51:54.546216+0000","last_active":"2024-09-21T22:51:54.545665+0000","last_peered":"2024-09-21T22:51:54.545665+0000","last_clean":"2024-09-21T22:51:54.545665+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:51:54.545665+0000","last_undegraded":"2024-09-21T22:51:54.545665+0000","last_fullsized":"2024-09-21T22:51:54.545665+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:31:36.528288+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079169670000000004,"stat_sum":{"num_bytes":34643350,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":11082,"num_read_kb":42973,"num_write":432,"num_write_kb":18958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.14","version":"100'431","reported_seq":9202,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.523481+0000","last_change":"2024-09-21T22:51:54.524807+0000","last_active":"2024-09-21T22:51:54.523481+0000","last_peered":"2024-09-21T22:51:54.523481+0000","last_clean":"2024-09-21T22:51:54.523481+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:51:54.523481+0000","last_undegraded":"2024-09-21T22:51:54.523481+0000","last_fullsized":"2024-09-21T22:51:54.523481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:25.248638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025072089999999998,"stat_sum":{"num_bytes":23449962,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":34,"num_read":9237,"num_read_kb":22032,"num_write":507,"num_write_kb":15782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T22:51:57.217343+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:51:57.217343+0000","last_peered":"2024-09-21T22:51:57.217343+0000","last_clean":"2024-09-21T22:51:57.217343+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:51:57.217343+0000","last_undegraded":"2024-09-21T22:51:57.217343+0000","last_fullsized":"2024-09-21T22:51:57.217343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"103'517","reported_seq":6455,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.526093+0000","last_change":"2024-09-21T22:51:54.526093+0000","last_active":"2024-09-21T22:51:54.526093+0000","last_peered":"2024-09-21T22:51:54.526093+0000","last_clean":"2024-09-21T22:51:54.526093+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:51:54.526093+0000","last_undegraded":"2024-09-21T22:51:54.526093+0000","last_fullsized":"2024-09-21T22:51:54.526093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:56:16.525522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024459299999999998,"stat_sum":{"num_bytes":16909325,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":31,"num_read":5985,"num_read_kb":22287,"num_write":505,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.653617+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:51:54.653617+0000","last_peered":"2024-09-21T22:51:54.653617+0000","last_clean":"2024-09-21T22:51:54.653617+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:51:54.653617+0000","last_undegraded":"2024-09-21T22:51:54.653617+0000","last_fullsized":"2024-09-21T22:51:54.653617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'442","reported_seq":4127,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.529963+0000","last_change":"2024-09-21T22:51:54.530260+0000","last_active":"2024-09-21T22:51:54.529963+0000","last_peered":"2024-09-21T22:51:54.529963+0000","last_clean":"2024-09-21T22:51:54.529963+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:51:54.529963+0000","last_undegraded":"2024-09-21T22:51:54.529963+0000","last_fullsized":"2024-09-21T22:51:54.529963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:03:14.670850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095798389999999997,"stat_sum":{"num_bytes":5871212,"num_objects":72,"num_object_clones":22,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":37,"num_read":4761,"num_read_kb":6226,"num_write":504,"num_write_kb":3540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":179,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.538663+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:51:54.538663+0000","last_peered":"2024-09-21T22:51:54.538663+0000","last_clean":"2024-09-21T22:51:54.538663+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:51:54.538663+0000","last_undegraded":"2024-09-21T22:51:54.538663+0000","last_fullsized":"2024-09-21T22:51:54.538663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"103'545","reported_seq":2502,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.659706+0000","last_change":"2024-09-21T22:51:54.659930+0000","last_active":"2024-09-21T22:51:54.659706+0000","last_peered":"2024-09-21T22:51:54.659706+0000","last_clean":"2024-09-21T22:51:54.659706+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:51:54.659706+0000","last_undegraded":"2024-09-21T22:51:54.659706+0000","last_fullsized":"2024-09-21T22:51:54.659706+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:43.003957+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0062888090000000002,"stat_sum":{"num_bytes":8336757,"num_objects":73,"num_object_clones":24,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":2835,"num_read_kb":6413,"num_write":662,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":179,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.653642+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:51:54.653642+0000","last_peered":"2024-09-21T22:51:54.653642+0000","last_clean":"2024-09-21T22:51:54.653642+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:51:54.653642+0000","last_undegraded":"2024-09-21T22:51:54.653642+0000","last_fullsized":"2024-09-21T22:51:54.653642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"105'542","reported_seq":2549,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.071610+0000","last_change":"2024-09-21T22:51:56.072154+0000","last_active":"2024-09-21T22:51:56.071610+0000","last_peered":"2024-09-21T22:51:56.071610+0000","last_clean":"2024-09-21T22:51:56.071610+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:51:56.071610+0000","last_undegraded":"2024-09-21T22:51:56.071610+0000","last_fullsized":"2024-09-21T22:51:56.071610+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:34:58.612956+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.024508652999999998,"stat_sum":{"num_bytes":7881347,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":27,"num_read":3528,"num_read_kb":13903,"num_write":536,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"7"}]},{"pgid":"3.3","version":"103'555","reported_seq":6238,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.683348+0000","last_change":"2024-09-21T22:51:54.527295+0000","last_active":"2024-09-21T22:51:54.683348+0000","last_peered":"2024-09-21T22:51:54.683348+0000","last_clean":"2024-09-21T22:51:54.683348+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:51:54.683348+0000","last_undegraded":"2024-09-21T22:51:54.683348+0000","last_fullsized":"2024-09-21T22:51:54.683348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:51.230963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044638669999999998,"stat_sum":{"num_bytes":15531119,"num_objects":64,"num_object_clones":21,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":32,"num_read":7840,"num_read_kb":13301,"num_write":532,"num_write_kb":6410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.2","version":"23'2","reported_seq":185,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.046616+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:51:56.046616+0000","last_peered":"2024-09-21T22:51:56.046616+0000","last_clean":"2024-09-21T22:51:56.046616+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:51:56.046616+0000","last_undegraded":"2024-09-21T22:51:56.046616+0000","last_fullsized":"2024-09-21T22:51:56.046616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"102'595","reported_seq":4595,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.654236+0000","last_change":"2024-09-21T22:51:54.654834+0000","last_active":"2024-09-21T22:51:54.654236+0000","last_peered":"2024-09-21T22:51:54.654236+0000","last_clean":"2024-09-21T22:51:54.654236+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:51:54.654236+0000","last_undegraded":"2024-09-21T22:51:54.654236+0000","last_fullsized":"2024-09-21T22:51:54.654236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:35:16.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011469570000000001,"stat_sum":{"num_bytes":23820633,"num_objects":78,"num_object_clones":27,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":37,"num_read":5808,"num_read_kb":19296,"num_write":891,"num_write_kb":12102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":186,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.519790+0000","last_change":"2024-09-21T22:51:42.453022+0000","last_active":"2024-09-21T22:51:54.519790+0000","last_peered":"2024-09-21T22:51:54.519790+0000","last_clean":"2024-09-21T22:51:54.519790+0000","last_became_active":"2024-09-21T22:51:42.452732+0000","last_became_peered":"2024-09-21T22:51:42.452732+0000","last_unstale":"2024-09-21T22:51:54.519790+0000","last_undegraded":"2024-09-21T22:51:54.519790+0000","last_fullsized":"2024-09-21T22:51:54.519790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:04.156384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'345","reported_seq":5544,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.524534+0000","last_change":"2024-09-21T22:51:54.525269+0000","last_active":"2024-09-21T22:51:54.524534+0000","last_peered":"2024-09-21T22:51:54.524534+0000","last_clean":"2024-09-21T22:51:54.524534+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:51:54.524534+0000","last_undegraded":"2024-09-21T22:51:54.524534+0000","last_fullsized":"2024-09-21T22:51:54.524534+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:24:56.353543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020174310000000001,"stat_sum":{"num_bytes":15074780,"num_objects":79,"num_object_clones":24,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":37,"num_read":5111,"num_read_kb":47271,"num_write":309,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.537858+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:51:54.537858+0000","last_peered":"2024-09-21T22:51:54.537858+0000","last_clean":"2024-09-21T22:51:54.537858+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:51:54.537858+0000","last_undegraded":"2024-09-21T22:51:54.537858+0000","last_fullsized":"2024-09-21T22:51:54.537858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"103'667","reported_seq":2951,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.673268+0000","last_change":"2024-09-21T22:51:54.533982+0000","last_active":"2024-09-21T22:51:54.673268+0000","last_peered":"2024-09-21T22:51:54.673268+0000","last_clean":"2024-09-21T22:51:54.673268+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:51:54.673268+0000","last_undegraded":"2024-09-21T22:51:54.673268+0000","last_fullsized":"2024-09-21T22:51:54.673268+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:17:24.402215+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013805992,"stat_sum":{"num_bytes":11862562,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":3952,"num_read_kb":14244,"num_write":831,"num_write_kb":10979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"2.3","version":"21'1","reported_seq":184,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.045862+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:51:56.045862+0000","last_peered":"2024-09-21T22:51:56.045862+0000","last_clean":"2024-09-21T22:51:56.045862+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:51:56.045862+0000","last_undegraded":"2024-09-21T22:51:56.045862+0000","last_fullsized":"2024-09-21T22:51:56.045862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":331,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.538103+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:51:54.538103+0000","last_peered":"2024-09-21T22:51:54.538103+0000","last_clean":"2024-09-21T22:51:54.538103+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:51:54.538103+0000","last_undegraded":"2024-09-21T22:51:54.538103+0000","last_fullsized":"2024-09-21T22:51:54.538103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"103'711","reported_seq":2763,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.733680+0000","last_change":"2024-09-21T22:51:54.528939+0000","last_active":"2024-09-21T22:51:54.733680+0000","last_peered":"2024-09-21T22:51:54.733680+0000","last_clean":"2024-09-21T22:51:54.733680+0000","last_became_active":"2024-09-21T22:51:45.471804+0000","last_became_peered":"2024-09-21T22:51:45.471804+0000","last_unstale":"2024-09-21T22:51:54.733680+0000","last_undegraded":"2024-09-21T22:51:54.733680+0000","last_fullsized":"2024-09-21T22:51:54.733680+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:02:35.807088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087152480000000001,"stat_sum":{"num_bytes":13445010,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":4188,"num_read_kb":7315,"num_write":956,"num_write_kb":5009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1097517,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.9","version":"103'770","reported_seq":5470,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.529889+0000","last_change":"2024-09-21T22:51:54.530460+0000","last_active":"2024-09-21T22:51:54.529889+0000","last_peered":"2024-09-21T22:51:54.529889+0000","last_clean":"2024-09-21T22:51:54.529889+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:51:54.529889+0000","last_undegraded":"2024-09-21T22:51:54.529889+0000","last_fullsized":"2024-09-21T22:51:54.529889+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:04:52.766833+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084558830000000008,"stat_sum":{"num_bytes":5970630,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":6840,"num_read_kb":10244,"num_write":976,"num_write_kb":5643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.a","version":"106'315","reported_seq":1304,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T22:51:57.219306+0000","last_change":"2024-09-21T22:51:57.219924+0000","last_active":"2024-09-21T22:51:57.219306+0000","last_peered":"2024-09-21T22:51:57.219306+0000","last_clean":"2024-09-21T22:51:57.219306+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:51:57.219306+0000","last_undegraded":"2024-09-21T22:51:57.219306+0000","last_fullsized":"2024-09-21T22:51:57.219306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:35.747429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024526349999999999,"stat_sum":{"num_bytes":3830200,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":896,"num_read_kb":14120,"num_write":295,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.b","version":"103'325","reported_seq":1434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.524836+0000","last_change":"2024-09-21T22:51:54.525201+0000","last_active":"2024-09-21T22:51:54.524836+0000","last_peered":"2024-09-21T22:51:54.524836+0000","last_clean":"2024-09-21T22:51:54.524836+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:51:54.524836+0000","last_undegraded":"2024-09-21T22:51:54.524836+0000","last_fullsized":"2024-09-21T22:51:54.524836+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:56:33.606278+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050787419999999998,"stat_sum":{"num_bytes":10954015,"num_objects":72,"num_object_clones":22,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":34,"num_read":1415,"num_read_kb":18056,"num_write":317,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.c","version":"105'302","reported_seq":5216,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T22:51:56.062027+0000","last_change":"2024-09-21T22:51:56.062587+0000","last_active":"2024-09-21T22:51:56.062027+0000","last_peered":"2024-09-21T22:51:56.062027+0000","last_clean":"2024-09-21T22:51:56.062027+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:51:56.062027+0000","last_undegraded":"2024-09-21T22:51:56.062027+0000","last_fullsized":"2024-09-21T22:51:56.062027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:21:15.646351+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.016017402,"stat_sum":{"num_bytes":15904705,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":4881,"num_read_kb":18947,"num_write":313,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"7"}]},{"pgid":"3.d","version":"99'570","reported_seq":3357,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.546358+0000","last_change":"2024-09-21T22:51:54.547046+0000","last_active":"2024-09-21T22:51:54.546358+0000","last_peered":"2024-09-21T22:51:54.546358+0000","last_clean":"2024-09-21T22:51:54.546358+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:51:54.546358+0000","last_undegraded":"2024-09-21T22:51:54.546358+0000","last_fullsized":"2024-09-21T22:51:54.546358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:57.548257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086932559999999999,"stat_sum":{"num_bytes":7892016,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":5329,"num_read_kb":8457,"num_write":538,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.e","version":"103'611","reported_seq":6344,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.687123+0000","last_change":"2024-09-21T22:51:54.547986+0000","last_active":"2024-09-21T22:51:54.687123+0000","last_peered":"2024-09-21T22:51:54.687123+0000","last_clean":"2024-09-21T22:51:54.687123+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:51:54.687123+0000","last_undegraded":"2024-09-21T22:51:54.687123+0000","last_fullsized":"2024-09-21T22:51:54.687123+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.914992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087138189999999994,"stat_sum":{"num_bytes":17008770,"num_objects":72,"num_object_clones":20,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":35,"num_read":7666,"num_read_kb":20511,"num_write":798,"num_write_kb":9358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.f","version":"103'406","reported_seq":5090,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.548775+0000","last_change":"2024-09-21T22:51:54.548775+0000","last_active":"2024-09-21T22:51:54.548775+0000","last_peered":"2024-09-21T22:51:54.548775+0000","last_clean":"2024-09-21T22:51:54.548775+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:51:54.548775+0000","last_undegraded":"2024-09-21T22:51:54.548775+0000","last_fullsized":"2024-09-21T22:51:54.548775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:52:19.814442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094785019999999998,"stat_sum":{"num_bytes":27316827,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5869,"num_read_kb":8766,"num_write":452,"num_write_kb":2859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.10","version":"105'571","reported_seq":7208,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T22:51:57.217570+0000","last_change":"2024-09-21T22:51:57.217862+0000","last_active":"2024-09-21T22:51:57.217570+0000","last_peered":"2024-09-21T22:51:57.217570+0000","last_clean":"2024-09-21T22:51:57.217570+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:51:57.217570+0000","last_undegraded":"2024-09-21T22:51:57.217570+0000","last_fullsized":"2024-09-21T22:51:57.217570+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:22.290010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045618500000000001,"stat_sum":{"num_bytes":19955667,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":34,"num_read":8437,"num_read_kb":38718,"num_write":580,"num_write_kb":9757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.11","version":"103'897","reported_seq":5666,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.546237+0000","last_change":"2024-09-21T22:51:54.547138+0000","last_active":"2024-09-21T22:51:54.546237+0000","last_peered":"2024-09-21T22:51:54.546237+0000","last_clean":"2024-09-21T22:51:54.546237+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:51:54.546237+0000","last_undegraded":"2024-09-21T22:51:54.546237+0000","last_fullsized":"2024-09-21T22:51:54.546237+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:02.382839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.008849506,"stat_sum":{"num_bytes":9020253,"num_objects":61,"num_object_clones":19,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":7278,"num_read_kb":52267,"num_write":1233,"num_write_kb":30194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.12","version":"100'475","reported_seq":8034,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.525406+0000","last_change":"2024-09-21T22:51:54.526066+0000","last_active":"2024-09-21T22:51:54.525406+0000","last_peered":"2024-09-21T22:51:54.525406+0000","last_clean":"2024-09-21T22:51:54.525406+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:51:54.525406+0000","last_undegraded":"2024-09-21T22:51:54.525406+0000","last_fullsized":"2024-09-21T22:51:54.525406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:06:39.866507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502644,"stat_sum":{"num_bytes":27922204,"num_objects":78,"num_object_clones":20,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":38,"num_read":9351,"num_read_kb":22697,"num_write":452,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.13","version":"103'530","reported_seq":7539,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.552794+0000","last_change":"2024-09-21T22:51:54.553261+0000","last_active":"2024-09-21T22:51:54.552794+0000","last_peered":"2024-09-21T22:51:54.552794+0000","last_clean":"2024-09-21T22:51:54.552794+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:51:54.552794+0000","last_undegraded":"2024-09-21T22:51:54.552794+0000","last_fullsized":"2024-09-21T22:51:54.552794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:33.980195+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014825797999999999,"stat_sum":{"num_bytes":28286437,"num_objects":90,"num_object_clones":28,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":48,"num_read":7382,"num_read_kb":28275,"num_write":609,"num_write_kb":14949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":509435957,"num_objects":2167,"num_object_clones":624,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":1079,"num_read":214292,"num_read_kb":711424,"num_write":22987,"num_write_kb":331596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237027328,"data_stored":272197466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":18846,"ondisk_log_size":18846,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476764,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":36496,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":37371904,"data_stored":45443734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87420,"kb_used_data":38184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94284420,"statfs":{"total":96636764160,"available":96547246080,"internally_reserved":0,"allocated":39100416,"data_stored":43422225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626450,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38536,"kb_used_data":6252,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94333304,"statfs":{"total":96636764160,"available":96597303296,"internally_reserved":0,"allocated":6402048,"data_stored":5829769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116616,"kb_used_data":41008,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94255224,"statfs":{"total":96636764160,"available":96517349376,"internally_reserved":0,"allocated":41992192,"data_stored":50853814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46316,"kb_used_data":6808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325524,"statfs":{"total":96636764160,"available":96589336576,"internally_reserved":0,"allocated":6971392,"data_stored":8398620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46568,"kb_used_data":15372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325272,"statfs":{"total":96636764160,"available":96589078528,"internally_reserved":0,"allocated":15740928,"data_stored":17416303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88968,"kb_used_data":31408,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94282872,"statfs":{"total":96636764160,"available":96545660928,"internally_reserved":0,"allocated":32161792,"data_stored":35206979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056644,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97260,"kb_used_data":67924,"kb_used_omap":42,"kb_used_meta":29269,"kb_avail":94274580,"statfs":{"total":96636764160,"available":96537169920,"internally_reserved":0,"allocated":69554176,"data_stored":72604440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":29972469},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30351360,"data_stored":34052248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13316096,"data_stored":15648939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5754880,"data_stored":7834257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":68333568,"data_stored":72039986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40775680,"data_stored":50289451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36044800,"data_stored":44812189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5160960,"data_stored":5252902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":37289984,"data_stored":42267494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:51:59.250 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [3, 6, 2, 4] 2024-09-21T22:51:59.250 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.8', '3', '6', '2', '4'] 2024-09-21T22:51:59.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.8 3 6 2 4 2024-09-21T22:51:59.251 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:51:59.248+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:51:59.297 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:59.297 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:59.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:51:59.352 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:51:59.347+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:51:59.453 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:59.452+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:59.467 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:59.468 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:59.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:51:59.554 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:59.551+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:59.583 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:51:59.649 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:51:59.650 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:51:59.655 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:59.655+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:59.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:51:59.712 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:51:59.712 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:51:59.727 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:51:59.728 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:51:59.743 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:51:59.743 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:51:59.756 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:51:59.751+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:51:59.759 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:51:59.759 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:51:59.779 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:51:59.779 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:51:59.796 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:59.796 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:59.796 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:51:59.796 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:59.797 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:51:59.798 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:51:59.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:51:59.816 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:51:59.816 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:51:59.836 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:51:59.837 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:51:59.857 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:51:59.856+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:51:59.958 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:51:59.956+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:51:59.980 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:51:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:51:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:51:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:51:59.980 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:51:59.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:00.051 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:52:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.8 pg_upmap_items mapping to [3->6,2->4] 2024-09-21T22:52:00.059 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:00.056+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:00.120 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138949045000004, 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:52:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:52:00.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:52:00.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138919044000005, 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:52:00.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138891337999993, 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.448 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:00.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:52:00.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:52:00.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138845387999993, 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:52:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138805253000001, 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:52:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:52:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138775253999995, 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138741193000001, 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:52:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138697338, 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138661206999998, 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:52:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138610626000002, 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:52:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138505412000001, 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138441901999997, 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:52:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:52:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138343699999993, 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:52:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:52:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138278436999997, 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138212519999996, 2024-09-21T22:52:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:52:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138135418000005, 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:52:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.138061817999997, 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:52:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.252960893999997, 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:52:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.974100317999998, 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 25.888341318999998, 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:52:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:00.524 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:00.525 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:00.526 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:00.523+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:00.627 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:00.623+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:00.700 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:00.700 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:00.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:00.728 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:00.728+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:00.828 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:00.828+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:00.880 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:00.880 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:00.880 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:00.880 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:00.880 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:00.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:00.942 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:00.939+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736735258000003, 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:52:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736693412000001, 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:52:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:52:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:52:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736647937999997, 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736613009999999, 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736580764000003, 2024-09-21T22:52:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736545032000002, 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:52:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736507856000003, 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.736473963000002, 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:52:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:52:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.728038619000003, 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:52:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:52:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.059021885999996, 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.061 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058970266999999, 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058918964, 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:52:01.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:52:01.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:52:01.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058861047999997, 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:52:01.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058805389, 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:01.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:52:01.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058749239999997, 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:52:01.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:01.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:52:01.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:52:01.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058686748, 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:52:01.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:52:01.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:52:01.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:52:01.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.05858593, 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:52:01.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:52:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058530111000003, 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.058477404999998, 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.510383732000001, 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:52:01.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.106 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:01.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:01.108 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:01.108+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:01.208 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:01.203+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:01.283 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:01.283 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:01.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:01.345 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:01.343+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:01.410 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:01.407+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:01.469 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:01.469 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:01.469 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:01.469 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:01.469 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:01.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:01.512 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:01.508+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:01.612 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:01.612+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:01.653 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 128.715210025, 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:01.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:01.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:52:01.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:01.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.660 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.170138166000001, 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:52:01.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.170074716000002, 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:52:01.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.170011537999997, 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:52:01.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169955197999997, 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169913456000003, 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:52:01.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:52:01.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169852855000002, 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169772418999997, 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:52:01.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169696309000003, 2024-09-21T22:52:01.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169609235999999, 2024-09-21T22:52:01.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:52:01.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:52:01.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169565747999997, 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:52:01.695 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:01.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169532110000006, 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169498344000004, 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:52:01.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169467060000002, 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:52:01.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:52:01.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169421705000005, 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169390208999999, 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:52:01.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.169355428000003, 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:52:01.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:52:01.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.151348853000002, 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:52:01.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.828768703000001, 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:01.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:52:01.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:52:01.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.127330514000001, 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:01.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:52:01.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:01.738 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:01.738 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:01.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:01.739 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:01.736+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:01.840 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:01.835+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:01.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:01.865 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:01.865 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:01.865 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:01.865 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:01.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:01.970 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:01.967+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:02.008 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:02.040 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:02.035+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:02.045 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:02.049 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:02.050 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:02.050 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:02.050 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:02.050 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:02.062 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:02.064 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:02.082 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:02.082 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:02.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:52:02.100 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:02.100 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:02.119 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:02.136 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:02.150 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:02.147+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:02.230 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:02.230 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:02.242 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:02.239+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:02.245 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:02.343 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:02.340+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:02.361 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:02.361 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:02.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:02.464 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:02.464+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:02.545 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:02.543+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:02.553 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:02.553 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:02.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:02.645 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:02.643+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:02.741 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:02.741 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:02.747 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:02.743+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:02.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:02.847 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:02.843+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:02.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:02.948 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:02.948+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:03.016 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.016 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:52:03.074 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:03.072+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:03.150 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:03.148+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:03.168 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.168 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:03.250 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:03.247+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:03.347 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.347 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.352 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:03.347+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:03.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:03.452 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:03.447+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:03.478 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.478 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:03.553 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:03.547+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:03.610 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.610 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:03.668 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:03.668+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:03.737 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.738 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.750 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:03.792 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:03.792+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:03.855 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:03.851+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:03.912 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:03.912 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:03.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:03.956 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:03.951+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:04.047 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:04.048 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:04.056 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:04.056+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:04.061 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:04.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:04.156 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:04.151+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:04.207 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:04.207 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:04.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:04.265 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:04.263+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:04.287 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:04.288 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:04.303 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:04.303 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:04.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:04.336 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:04.336 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:04.355 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:04.358 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:04.356+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:04.375 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:04.376 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:04.388 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:04.388 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:04.388 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:04.388 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:04.388 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:04.396 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:04.396 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:04.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:04.414 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:04.414 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:04.472 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:04.472+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:04.535 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:04.535 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552739509999995, 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:52:04.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552709508999996, 2024-09-21T22:52:04.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:04.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552681802999999, 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:04.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552635852999998, 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552595718000006, 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552565719, 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:52:04.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:52:04.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552531658000007, 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:52:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:52:04.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552487803000005, 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:52:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:52:04.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:52:04.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552451672000004, 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:52:04.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:52:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:52:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552401090999993, 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:52:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:52:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552295877000006, 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552232367000002, 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:52:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:52:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552134164999998, 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:52:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:52:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552068902000002, 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:52:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.552002985000001, 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:52:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.551925882999996, 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:52:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.551852283000002, 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:52:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:52:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.666751359000003, 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.387890783000003, 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.302131784, 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:52:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:04.621 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:04.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:04.623 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:04.620+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:04.723 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:04.720+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:04.749 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:04.749 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:04.749 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:04.749 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:04.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:04.823 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:04.820+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:04.884 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:04.884 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:04.884 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:04.884 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:04.884 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:04.898 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:04.946 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:04.943+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:05.025 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:05.024+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.747163155000003, 2024-09-21T22:52:05.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:52:05.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:52:05.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:52:05.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.747121309000001, 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:52:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:52:05.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:52:05.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.747075834999997, 2024-09-21T22:52:05.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:52:05.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:52:05.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:52:05.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.747040906999999, 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:52:05.042 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.747008661000002, 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.746972929000002, 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.052 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:05.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.746935753000002, 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.746901860000001, 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.738466516000003, 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:52:05.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069449783000003, 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:52:05.071 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:52:05.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069398163999999, 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:52:05.075 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.079 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069346861, 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069288944999997, 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069233285999999, 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:05.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:52:05.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069177136999997, 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069114644999999, 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:05.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.069013826999999, 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.104 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.068958008000003, 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:05.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.068905301999997, 2024-09-21T22:52:05.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:52:05.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:52:05.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:05.112 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.520811629000001, 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:52:05.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:05.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.118 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:05.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:52:05.119 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 7, 6, 3] 2024-09-21T22:52:05.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:52:05.119 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T22:52:05.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T22:52:05.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:05.168 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:05.168+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:05.226 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:05.223+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:05.303 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:05.303 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:05.303 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:05.317 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:05.327 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:05.324+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:05.427 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:05.424+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:05.451 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:05.451 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:05.451 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 114, 2024-09-21T22:52:05.451 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "56013142-0739-4aee-ba29-30eddcc2dbe0", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-21T22:49:36.309693+0000", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-21T22:52:05.068044+0000", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-21T22:51:41.430550+0000", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-21T22:49:39.319546+0000", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 7, 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T22:52:05.452 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:51.363777+0000", 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-21T22:52:05.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-21T22:52:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "19", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:52:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:52:05.456 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:52:05.457 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-21T22:52:05.458 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:49:55.226638+0000", 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:52:05.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-21T22:52:05.460 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "23", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:52:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 23, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:52:05.462 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:52:05.463 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-21T22:52:05.464 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-21T22:50:05.084190+0000", 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-21T22:52:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-21T22:52:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "114", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 37, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 114, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-21T22:52:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.468 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-21T22:52:05.469 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.5, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.5, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.5, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.5, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.470 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ecfcea04-7198-4c57-bc74-45aaf791e3b7", 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 78, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.471 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211646084 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/2211646084", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/2211646084", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/2211646084", 2024-09-21T22:52:05.475 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/2211646084", 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b6b749e6-4a66-4d8b-bf50-c9634a6927d5", 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:52:05.476 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 111, 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.477 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.479 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1102146551 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1102146551", 2024-09-21T22:52:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1102146551", 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1102146551", 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1102146551", 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "4e90da65-2fdd-4c85-a207-70d5f22c95a4", 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.481 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 90, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-21T22:52:05.485 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2482161893 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/2482161893", 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/2482161893", 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/2482161893", 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/2482161893", 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.486 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "3c97651a-402c-4171-9648-c615dd754c94", 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 13, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 77, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 90, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 93, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 78, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.487 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-21T22:52:05.490 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2575188997 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/2575188997", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/2575188997", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/2575188997", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/2575188997", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b935b00e-0d91-4f2b-9f2d-2c2be1144ca8", 2024-09-21T22:52:05.491 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 24, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6819", 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6821", 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6823", 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6826", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6832", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6833", 2024-09-21T22:52:05.494 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6828", 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6830", 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1105568802 2024-09-21T22:52:05.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6821/1105568802", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6826/1105568802", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6833/1105568802", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6830/1105568802", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "559dea61-5abd-423e-9dfb-cf92f9e6d4a4", 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.496 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 16, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 93, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6818", 2024-09-21T22:52:05.497 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6820", 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6822", 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6824", 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6829", 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6831", 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6825", 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.500 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6827", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2691947692 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6820/2691947692", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6824/2691947692", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2691947692", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6827/2691947692", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:52:05.501 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "aeee922d-53c2-41f4-9433-8a9691f3e674", 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 15, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 23, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 42, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 112, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 24, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.502 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6810", 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6811", 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6812", 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6813", 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6816", 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6817", 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6814", 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.505 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6815", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3470582226 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6811/3470582226", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6813/3470582226", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6817/3470582226", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6815/3470582226", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.506 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "b40a3234-6950-4a73-9a5c-f0c086450816", 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 90, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6800", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6801", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6802", 2024-09-21T22:52:05.508 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6803", 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6806", 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6807", 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6804", 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.99:6805", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1713641021 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.99:6801/1713641021", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.99:6803/1713641021", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1713641021", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1713641021", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.511 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.846676+0000", 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:47.117051+0000", 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.836357+0000", 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.351920+0000", 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 78 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.514 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.975489+0000", 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.838876+0000", 2024-09-21T22:52:05.515 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.842751+0000", 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 24 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-21T22:52:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540701547738038271, 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-21T22:49:46.784805+0000", 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [ 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.1f", 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: 3, 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: 5 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.8", 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 6 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-21T22:52:05.518 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 20, 2024-09-21T22:52:05.519 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 9 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 20, 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-21T22:52:05.520 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:05.521 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:05.522 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:05.523 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:05.523 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:05.523 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:05.523 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:05.523 INFO:tasks.thrashosds.thrasher:j is {'epoch': 114, 'fsid': '56013142-0739-4aee-ba29-30eddcc2dbe0', 'created': '2024-09-21T22:49:36.309693+0000', 'modified': '2024-09-21T22:52:05.068044+0000', 'last_up_change': '2024-09-21T22:51:41.430550+0000', 'last_in_change': '2024-09-21T22:49:39.319546+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T22:49:51.363777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T22:49:55.226638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T22:50:05.084190+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ecfcea04-7198-4c57-bc74-45aaf791e3b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 78, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 2211646084}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 2211646084}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 2211646084}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 2211646084}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 2211646084}]}, 'public_addr': '172.21.15.94:6809/2211646084', 'cluster_addr': '172.21.15.94:6811/2211646084', 'heartbeat_back_addr': '172.21.15.94:6815/2211646084', 'heartbeat_front_addr': '172.21.15.94:6813/2211646084', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b6b749e6-4a66-4d8b-bf50-c9634a6927d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1102146551}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1102146551}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1102146551}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1102146551}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1102146551}]}, 'public_addr': '172.21.15.94:6825/1102146551', 'cluster_addr': '172.21.15.94:6827/1102146551', 'heartbeat_back_addr': '172.21.15.94:6831/1102146551', 'heartbeat_front_addr': '172.21.15.94:6829/1102146551', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4e90da65-2fdd-4c85-a207-70d5f22c95a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 2482161893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 2482161893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 2482161893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 2482161893}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 2482161893}]}, 'public_addr': '172.21.15.94:6817/2482161893', 'cluster_addr': '172.21.15.94:6819/2482161893', 'heartbeat_back_addr': '172.21.15.94:6823/2482161893', 'heartbeat_front_addr': '172.21.15.94:6821/2482161893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3c97651a-402c-4171-9648-c615dd754c94', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 77, 'up_from': 90, 'up_thru': 93, 'down_at': 78, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 2575188997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 2575188997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 2575188997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 2575188997}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 2575188997}]}, 'public_addr': '172.21.15.94:6801/2575188997', 'cluster_addr': '172.21.15.94:6803/2575188997', 'heartbeat_back_addr': '172.21.15.94:6807/2575188997', 'heartbeat_front_addr': '172.21.15.94:6805/2575188997', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b935b00e-0d91-4f2b-9f2d-2c2be1144ca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6819', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 1105568802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6823', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6826', 'nonce': 1105568802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 1105568802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 1105568802}, {'type': 'v1', 'addr': '172.21.15.99:6830', 'nonce': 1105568802}]}, 'public_addr': '172.21.15.99:6821/1105568802', 'cluster_addr': '172.21.15.99:6826/1105568802', 'heartbeat_back_addr': '172.21.15.99:6833/1105568802', 'heartbeat_front_addr': '172.21.15.99:6830/1105568802', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '559dea61-5abd-423e-9dfb-cf92f9e6d4a4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6820', 'nonce': 2691947692}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6824', 'nonce': 2691947692}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6829', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2691947692}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6825', 'nonce': 2691947692}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2691947692}]}, 'public_addr': '172.21.15.99:6820/2691947692', 'cluster_addr': '172.21.15.99:6824/2691947692', 'heartbeat_back_addr': '172.21.15.99:6831/2691947692', 'heartbeat_front_addr': '172.21.15.99:6827/2691947692', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'aeee922d-53c2-41f4-9433-8a9691f3e674', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 23, 'up_from': 42, 'up_thru': 112, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3470582226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 3470582226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3470582226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 3470582226}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3470582226}]}, 'public_addr': '172.21.15.99:6811/3470582226', 'cluster_addr': '172.21.15.99:6813/3470582226', 'heartbeat_back_addr': '172.21.15.99:6817/3470582226', 'heartbeat_front_addr': '172.21.15.99:6815/3470582226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b40a3234-6950-4a73-9a5c-f0c086450816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1713641021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1713641021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1713641021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1713641021}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1713641021}]}, 'public_addr': '172.21.15.99:6801/1713641021', 'cluster_addr': '172.21.15.99:6803/1713641021', 'heartbeat_back_addr': '172.21.15.99:6807/1713641021', 'heartbeat_front_addr': '172.21.15.99:6805/1713641021', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.846676+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:47.117051+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.836357+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.351920+0000', 'dead_epoch': 78}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.975489+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.838876+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.842751+0000', 'dead_epoch': 24}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T22:49:46.784805+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1f', 'osds': [3, 5]}], 'pg_upmap_items': [{'pgid': '3.8', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T22:52:05.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:05.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:05.569 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:05.566+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:05.629 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:05.622+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:05.647 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:05.647 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:05.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:05.647 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:05.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 132.71048989299999, 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:52:05.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165418033999998, 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:52:05.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:52:05.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165354583999999, 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:52:05.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165291405999994, 2024-09-21T22:52:05.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:52:05.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:52:05.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.667 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165235065999994, 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165193324000001, 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:52:05.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165132722999999, 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:05.680 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.165052286999995, 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:52:05.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:52:05.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164976177, 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:52:05.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:52:05.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:52:05.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164889103999997, 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164845615999994, 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:52:05.693 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:52:05.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:52:05.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164811978000003, 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164778212000002, 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.706 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164746928, 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:52:05.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:52:05.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164701573000002, 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164670076999997, 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:05.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:52:05.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.164635296, 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:05.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:52:05.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.146628720999999, 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:52:05.723 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.824048570999999, 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:52:05.727 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:52:05.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:52:05.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:52:05.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:52:05.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.122610382000001, 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:05.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:05.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:05.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:05.739 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:05.740 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:05.741 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:05.740+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:05.841 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:05.840+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:05.871 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:05.872 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:05.890 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-21T22:52:05.305936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355538837,"num_objects":1918,"num_object_clones":547,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":1013,"num_read":215820,"num_read_kb":712729,"num_write":23441,"num_write_kb":334086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19434,"ondisk_log_size":19434,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":568132,"kb_used_data":189456,"kb_used_omap":55,"kb_used_meta":376200,"kb_avail":754406588,"statfs":{"total":773094113280,"available":772512346112,"internally_reserved":0,"allocated":194002944,"data_stored":217876121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":385229162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25101937,"num_objects":-36,"num_object_clones":-11,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":-9,"num_read":63,"num_read_kb":54,"num_write":42,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000995"},"pg_stats":[{"pgid":"3.1f","version":"109'966","reported_seq":7162,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.067531+0000","last_change":"2024-09-21T22:52:00.068037+0000","last_active":"2024-09-21T22:52:00.067531+0000","last_peered":"2024-09-21T22:52:00.067531+0000","last_clean":"2024-09-21T22:52:00.067531+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:00.067531+0000","last_undegraded":"2024-09-21T22:52:00.067531+0000","last_fullsized":"2024-09-21T22:52:00.067531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:22:15.215622+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011677195,"stat_sum":{"num_bytes":12229352,"num_objects":70,"num_object_clones":23,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":32,"num_read":9832,"num_read_kb":21964,"num_write":1248,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1e","version":"109'520","reported_seq":6029,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.222984+0000","last_change":"2024-09-21T22:52:00.066029+0000","last_active":"2024-09-21T22:52:00.222984+0000","last_peered":"2024-09-21T22:52:00.222984+0000","last_clean":"2024-09-21T22:52:00.222984+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:00.222984+0000","last_undegraded":"2024-09-21T22:52:00.222984+0000","last_fullsized":"2024-09-21T22:52:00.222984+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:08.560008+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0096715710000000003,"stat_sum":{"num_bytes":9433572,"num_objects":69,"num_object_clones":15,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":38,"num_read":7349,"num_read_kb":20474,"num_write":478,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1d","version":"110'1009","reported_seq":7195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.083279+0000","last_change":"2024-09-21T22:52:01.083472+0000","last_active":"2024-09-21T22:52:01.083279+0000","last_peered":"2024-09-21T22:52:01.083279+0000","last_clean":"2024-09-21T22:52:01.083279+0000","last_became_active":"2024-09-21T22:50:07.036624+0000","last_became_peered":"2024-09-21T22:50:07.036624+0000","last_unstale":"2024-09-21T22:52:01.083279+0000","last_undegraded":"2024-09-21T22:52:01.083279+0000","last_fullsized":"2024-09-21T22:52:01.083279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:06:57.196561+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013759904,"stat_sum":{"num_bytes":25429419,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":32,"num_read":10438,"num_read_kb":47237,"num_write":1476,"num_write_kb":18443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.1c","version":"110'779","reported_seq":2127,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.133030+0000","last_change":"2024-09-21T22:52:01.070220+0000","last_active":"2024-09-21T22:52:01.133030+0000","last_peered":"2024-09-21T22:52:01.133030+0000","last_clean":"2024-09-21T22:52:01.133030+0000","last_became_active":"2024-09-21T22:50:07.036838+0000","last_became_peered":"2024-09-21T22:50:07.036838+0000","last_unstale":"2024-09-21T22:52:01.133030+0000","last_undegraded":"2024-09-21T22:52:01.133030+0000","last_fullsized":"2024-09-21T22:52:01.133030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:24.528520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011258069999999999,"stat_sum":{"num_bytes":4698262,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":34,"num_read":3538,"num_read_kb":10901,"num_write":1051,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.1b","version":"109'1045","reported_seq":6474,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.177060+0000","last_change":"2024-09-21T22:52:00.059422+0000","last_active":"2024-09-21T22:52:00.177060+0000","last_peered":"2024-09-21T22:52:00.177060+0000","last_clean":"2024-09-21T22:52:00.177060+0000","last_became_active":"2024-09-21T22:50:07.036599+0000","last_became_peered":"2024-09-21T22:50:07.036599+0000","last_unstale":"2024-09-21T22:52:00.177060+0000","last_undegraded":"2024-09-21T22:52:00.177060+0000","last_fullsized":"2024-09-21T22:52:00.177060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:48:20.622618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025065360000000002,"stat_sum":{"num_bytes":6285025,"num_objects":56,"num_object_clones":20,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":9592,"num_read_kb":37832,"num_write":1597,"num_write_kb":12283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1a","version":"108'752","reported_seq":8435,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.200147+0000","last_change":"2024-09-21T22:52:00.055839+0000","last_active":"2024-09-21T22:52:00.200147+0000","last_peered":"2024-09-21T22:52:00.200147+0000","last_clean":"2024-09-21T22:52:00.200147+0000","last_became_active":"2024-09-21T22:50:07.037317+0000","last_became_peered":"2024-09-21T22:50:07.037317+0000","last_unstale":"2024-09-21T22:52:00.200147+0000","last_undegraded":"2024-09-21T22:52:00.200147+0000","last_fullsized":"2024-09-21T22:52:00.200147+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:25.523053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071188600000000001,"stat_sum":{"num_bytes":8312382,"num_objects":64,"num_object_clones":22,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":29,"num_read":9884,"num_read_kb":13050,"num_write":1096,"num_write_kb":4809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.19","version":"109'485","reported_seq":8440,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.220003+0000","last_change":"2024-09-21T22:52:00.063384+0000","last_active":"2024-09-21T22:52:00.220003+0000","last_peered":"2024-09-21T22:52:00.220003+0000","last_clean":"2024-09-21T22:52:00.220003+0000","last_became_active":"2024-09-21T22:51:42.461781+0000","last_became_peered":"2024-09-21T22:51:42.461781+0000","last_unstale":"2024-09-21T22:52:00.220003+0000","last_undegraded":"2024-09-21T22:52:00.220003+0000","last_fullsized":"2024-09-21T22:52:00.220003+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:46:43.374803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058664299999999997,"stat_sum":{"num_bytes":10407981,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":8676,"num_read_kb":60916,"num_write":499,"num_write_kb":24712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":20755516,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.18","version":"109'1052","reported_seq":11171,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.074840+0000","last_change":"2024-09-21T22:52:00.074840+0000","last_active":"2024-09-21T22:52:00.074840+0000","last_peered":"2024-09-21T22:52:00.074840+0000","last_clean":"2024-09-21T22:52:00.074840+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:00.074840+0000","last_undegraded":"2024-09-21T22:52:00.074840+0000","last_fullsized":"2024-09-21T22:52:00.074840+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:12.401705+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018210213999999999,"stat_sum":{"num_bytes":8929620,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":14214,"num_read_kb":27572,"num_write":1660,"num_write_kb":11377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.17","version":"109'466","reported_seq":3282,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.224076+0000","last_change":"2024-09-21T22:52:00.060031+0000","last_active":"2024-09-21T22:52:00.224076+0000","last_peered":"2024-09-21T22:52:00.224076+0000","last_clean":"2024-09-21T22:52:00.224076+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:00.224076+0000","last_undegraded":"2024-09-21T22:52:00.224076+0000","last_fullsized":"2024-09-21T22:52:00.224076+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:27:39.732748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031106139999999998,"stat_sum":{"num_bytes":4798614,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":33,"num_read":3823,"num_read_kb":6407,"num_write":429,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.16","version":"108'394","reported_seq":8525,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.070256+0000","last_change":"2024-09-21T22:52:01.070696+0000","last_active":"2024-09-21T22:52:01.070256+0000","last_peered":"2024-09-21T22:52:01.070256+0000","last_clean":"2024-09-21T22:52:01.070256+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:01.070256+0000","last_undegraded":"2024-09-21T22:52:01.070256+0000","last_fullsized":"2024-09-21T22:52:01.070256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:02.658874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037846,"stat_sum":{"num_bytes":14048245,"num_objects":66,"num_object_clones":19,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":37,"num_read":7939,"num_read_kb":9321,"num_write":336,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.15","version":"109'431","reported_seq":11197,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.075266+0000","last_change":"2024-09-21T22:52:00.075667+0000","last_active":"2024-09-21T22:52:00.075266+0000","last_peered":"2024-09-21T22:52:00.075266+0000","last_clean":"2024-09-21T22:52:00.075266+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:00.075266+0000","last_undegraded":"2024-09-21T22:52:00.075266+0000","last_fullsized":"2024-09-21T22:52:00.075266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:26.940665+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.014435445,"stat_sum":{"num_bytes":6378672,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":11090,"num_read_kb":42977,"num_write":439,"num_write_kb":18958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.14","version":"108'459","reported_seq":9273,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.055926+0000","last_change":"2024-09-21T22:52:00.056239+0000","last_active":"2024-09-21T22:52:00.055926+0000","last_peered":"2024-09-21T22:52:00.055926+0000","last_clean":"2024-09-21T22:52:00.055926+0000","last_became_active":"2024-09-21T22:50:07.030600+0000","last_became_peered":"2024-09-21T22:50:07.030600+0000","last_unstale":"2024-09-21T22:52:00.055926+0000","last_undegraded":"2024-09-21T22:52:00.055926+0000","last_fullsized":"2024-09-21T22:52:00.055926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:27.799895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077412399999999995,"stat_sum":{"num_bytes":16522892,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":9268,"num_read_kb":22056,"num_write":524,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":161,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.044381+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:01.044381+0000","last_peered":"2024-09-21T22:52:01.044381+0000","last_clean":"2024-09-21T22:52:01.044381+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:01.044381+0000","last_undegraded":"2024-09-21T22:52:01.044381+0000","last_fullsized":"2024-09-21T22:52:01.044381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"109'533","reported_seq":6513,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.067809+0000","last_change":"2024-09-21T22:52:00.067809+0000","last_active":"2024-09-21T22:52:00.067809+0000","last_peered":"2024-09-21T22:52:00.067809+0000","last_clean":"2024-09-21T22:52:00.067809+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:00.067809+0000","last_undegraded":"2024-09-21T22:52:00.067809+0000","last_fullsized":"2024-09-21T22:52:00.067809+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:00:10.681762+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010441354,"stat_sum":{"num_bytes":6718228,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":28,"num_read":5987,"num_read_kb":22288,"num_write":509,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.062391+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:00.062391+0000","last_peered":"2024-09-21T22:52:00.062391+0000","last_clean":"2024-09-21T22:52:00.062391+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:00.062391+0000","last_undegraded":"2024-09-21T22:52:00.062391+0000","last_fullsized":"2024-09-21T22:52:00.062391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"108'449","reported_seq":4176,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.066243+0000","last_change":"2024-09-21T22:52:00.068694+0000","last_active":"2024-09-21T22:52:00.066243+0000","last_peered":"2024-09-21T22:52:00.066243+0000","last_clean":"2024-09-21T22:52:00.066243+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:00.066243+0000","last_undegraded":"2024-09-21T22:52:00.066243+0000","last_fullsized":"2024-09-21T22:52:00.066243+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:34.567685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012151959,"stat_sum":{"num_bytes":4281964,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":4761,"num_read_kb":6226,"num_write":507,"num_write_kb":3540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.061714+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:00.061714+0000","last_peered":"2024-09-21T22:52:00.061714+0000","last_clean":"2024-09-21T22:52:00.061714+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:00.061714+0000","last_undegraded":"2024-09-21T22:52:00.061714+0000","last_fullsized":"2024-09-21T22:52:00.061714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"109'562","reported_seq":2564,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.168279+0000","last_change":"2024-09-21T22:52:00.063242+0000","last_active":"2024-09-21T22:52:00.168279+0000","last_peered":"2024-09-21T22:52:00.168279+0000","last_clean":"2024-09-21T22:52:00.168279+0000","last_became_active":"2024-09-21T22:50:07.220301+0000","last_became_peered":"2024-09-21T22:50:07.220301+0000","last_unstale":"2024-09-21T22:52:00.168279+0000","last_undegraded":"2024-09-21T22:52:00.168279+0000","last_fullsized":"2024-09-21T22:52:00.168279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:31:46.289229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087657499999999999,"stat_sum":{"num_bytes":7725088,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":2854,"num_read_kb":6427,"num_write":672,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.062543+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:00.062543+0000","last_peered":"2024-09-21T22:52:00.062543+0000","last_clean":"2024-09-21T22:52:00.062543+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:00.062543+0000","last_undegraded":"2024-09-21T22:52:00.062543+0000","last_fullsized":"2024-09-21T22:52:00.062543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'547","reported_seq":2589,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.084349+0000","last_change":"2024-09-21T22:52:01.084522+0000","last_active":"2024-09-21T22:52:01.084349+0000","last_peered":"2024-09-21T22:52:01.084349+0000","last_clean":"2024-09-21T22:52:01.084349+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:01.084349+0000","last_undegraded":"2024-09-21T22:52:01.084349+0000","last_fullsized":"2024-09-21T22:52:01.084349+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:26.580972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014593350999999999,"stat_sum":{"num_bytes":7223410,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":3528,"num_read_kb":13903,"num_write":539,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.3","version":"109'586","reported_seq":6335,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.187836+0000","last_change":"2024-09-21T22:52:00.056703+0000","last_active":"2024-09-21T22:52:00.187836+0000","last_peered":"2024-09-21T22:52:00.187836+0000","last_clean":"2024-09-21T22:52:00.187836+0000","last_became_active":"2024-09-21T22:50:07.037995+0000","last_became_peered":"2024-09-21T22:50:07.037995+0000","last_unstale":"2024-09-21T22:52:00.187836+0000","last_undegraded":"2024-09-21T22:52:00.187836+0000","last_fullsized":"2024-09-21T22:52:00.187836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:46:30.527538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140039,"stat_sum":{"num_bytes":14060655,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":7876,"num_read_kb":13337,"num_write":559,"num_write_kb":6428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.2","version":"23'2","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.068993+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:01.068993+0000","last_peered":"2024-09-21T22:52:01.068993+0000","last_clean":"2024-09-21T22:52:01.068993+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:01.068993+0000","last_undegraded":"2024-09-21T22:52:01.068993+0000","last_fullsized":"2024-09-21T22:52:01.068993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"105'599","reported_seq":4640,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.064802+0000","last_change":"2024-09-21T22:52:00.065403+0000","last_active":"2024-09-21T22:52:00.064802+0000","last_peered":"2024-09-21T22:52:00.064802+0000","last_clean":"2024-09-21T22:52:00.064802+0000","last_became_active":"2024-09-21T22:50:07.221714+0000","last_became_peered":"2024-09-21T22:50:07.221714+0000","last_unstale":"2024-09-21T22:52:00.064802+0000","last_undegraded":"2024-09-21T22:52:00.064802+0000","last_fullsized":"2024-09-21T22:52:00.064802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:36.572201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012650370000000001,"stat_sum":{"num_bytes":23714137,"num_objects":74,"num_object_clones":25,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":5808,"num_read_kb":19296,"num_write":891,"num_write_kb":12102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":198,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.073547+0000","last_change":"2024-09-21T22:51:42.453022+0000","last_active":"2024-09-21T22:52:00.073547+0000","last_peered":"2024-09-21T22:52:00.073547+0000","last_clean":"2024-09-21T22:52:00.073547+0000","last_became_active":"2024-09-21T22:51:42.452732+0000","last_became_peered":"2024-09-21T22:51:42.452732+0000","last_unstale":"2024-09-21T22:52:00.073547+0000","last_undegraded":"2024-09-21T22:52:00.073547+0000","last_fullsized":"2024-09-21T22:52:00.073547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:04.156384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"108'352","reported_seq":5593,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.057173+0000","last_change":"2024-09-21T22:52:00.057626+0000","last_active":"2024-09-21T22:52:00.057173+0000","last_peered":"2024-09-21T22:52:00.057173+0000","last_clean":"2024-09-21T22:52:00.057173+0000","last_became_active":"2024-09-21T22:50:07.041186+0000","last_became_peered":"2024-09-21T22:50:07.041186+0000","last_unstale":"2024-09-21T22:52:00.057173+0000","last_undegraded":"2024-09-21T22:52:00.057173+0000","last_fullsized":"2024-09-21T22:52:00.057173+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:36.769596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000894135,"stat_sum":{"num_bytes":13755868,"num_objects":72,"num_object_clones":23,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":36,"num_read":5111,"num_read_kb":47271,"num_write":314,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.061016+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:00.061016+0000","last_peered":"2024-09-21T22:52:00.061016+0000","last_clean":"2024-09-21T22:52:00.061016+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:00.061016+0000","last_undegraded":"2024-09-21T22:52:00.061016+0000","last_fullsized":"2024-09-21T22:52:00.061016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"108'669","reported_seq":2995,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.057019+0000","last_change":"2024-09-21T22:52:00.058212+0000","last_active":"2024-09-21T22:52:00.057019+0000","last_peered":"2024-09-21T22:52:00.057019+0000","last_clean":"2024-09-21T22:52:00.057019+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:00.057019+0000","last_undegraded":"2024-09-21T22:52:00.057019+0000","last_fullsized":"2024-09-21T22:52:00.057019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:46:12.827455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002151746,"stat_sum":{"num_bytes":11584034,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":32,"num_read":3952,"num_read_kb":14244,"num_write":833,"num_write_kb":10979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.3","version":"21'1","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.068995+0000","last_change":"2024-09-21T22:49:58.991819+0000","last_active":"2024-09-21T22:52:01.068995+0000","last_peered":"2024-09-21T22:52:01.068995+0000","last_clean":"2024-09-21T22:52:01.068995+0000","last_became_active":"2024-09-21T22:49:56.978807+0000","last_became_peered":"2024-09-21T22:49:56.978807+0000","last_unstale":"2024-09-21T22:52:01.068995+0000","last_undegraded":"2024-09-21T22:52:01.068995+0000","last_fullsized":"2024-09-21T22:52:01.068995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:10.340559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":343,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.061113+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:00.061113+0000","last_peered":"2024-09-21T22:52:00.061113+0000","last_clean":"2024-09-21T22:52:00.061113+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:00.061113+0000","last_undegraded":"2024-09-21T22:52:00.061113+0000","last_fullsized":"2024-09-21T22:52:00.061113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"103'711","reported_seq":2763,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-21T22:51:54.733680+0000","last_change":"2024-09-21T22:51:54.528939+0000","last_active":"2024-09-21T22:51:54.733680+0000","last_peered":"2024-09-21T22:51:54.733680+0000","last_clean":"2024-09-21T22:51:54.733680+0000","last_became_active":"2024-09-21T22:51:45.471804+0000","last_became_peered":"2024-09-21T22:51:45.471804+0000","last_unstale":"2024-09-21T22:51:54.733680+0000","last_undegraded":"2024-09-21T22:51:54.733680+0000","last_fullsized":"2024-09-21T22:51:54.733680+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:02:35.807088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087152480000000001,"stat_sum":{"num_bytes":13445010,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":4188,"num_read_kb":7315,"num_write":956,"num_write_kb":5009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1097517,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"9"}]},{"pgid":"3.9","version":"108'777","reported_seq":5521,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.055601+0000","last_change":"2024-09-21T22:52:00.055904+0000","last_active":"2024-09-21T22:52:00.055601+0000","last_peered":"2024-09-21T22:52:00.055601+0000","last_clean":"2024-09-21T22:52:00.055601+0000","last_became_active":"2024-09-21T22:50:07.223082+0000","last_became_peered":"2024-09-21T22:50:07.223082+0000","last_unstale":"2024-09-21T22:52:00.055601+0000","last_undegraded":"2024-09-21T22:52:00.055601+0000","last_fullsized":"2024-09-21T22:52:00.055601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:33:32.091076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073593499999999995,"stat_sum":{"num_bytes":4577974,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":36,"num_read":6843,"num_read_kb":10245,"num_write":979,"num_write_kb":5643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.a","version":"110'320","reported_seq":1338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.045086+0000","last_change":"2024-09-21T22:52:01.045086+0000","last_active":"2024-09-21T22:52:01.045086+0000","last_peered":"2024-09-21T22:52:01.045086+0000","last_clean":"2024-09-21T22:52:01.045086+0000","last_became_active":"2024-09-21T22:50:34.191905+0000","last_became_peered":"2024-09-21T22:50:34.191905+0000","last_unstale":"2024-09-21T22:52:01.045086+0000","last_undegraded":"2024-09-21T22:52:01.045086+0000","last_fullsized":"2024-09-21T22:52:01.045086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:36.309593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059974300000000002,"stat_sum":{"num_bytes":3412408,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":899,"num_read_kb":14123,"num_write":300,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":35205,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.b","version":"108'336","reported_seq":1486,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.056892+0000","last_change":"2024-09-21T22:52:00.058586+0000","last_active":"2024-09-21T22:52:00.056892+0000","last_peered":"2024-09-21T22:52:00.056892+0000","last_clean":"2024-09-21T22:52:00.056892+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:00.056892+0000","last_undegraded":"2024-09-21T22:52:00.056892+0000","last_fullsized":"2024-09-21T22:52:00.056892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:24:02.848664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025392919999999999,"stat_sum":{"num_bytes":5767083,"num_objects":61,"num_object_clones":20,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":32,"num_read":1415,"num_read_kb":18056,"num_write":324,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.c","version":"110'320","reported_seq":5272,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.080134+0000","last_change":"2024-09-21T22:52:01.080894+0000","last_active":"2024-09-21T22:52:01.080134+0000","last_peered":"2024-09-21T22:52:01.080134+0000","last_clean":"2024-09-21T22:52:01.080134+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:01.080134+0000","last_undegraded":"2024-09-21T22:52:01.080134+0000","last_fullsized":"2024-09-21T22:52:01.080134+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:05.952522+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.011362293000000001,"stat_sum":{"num_bytes":5868377,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":25,"num_read":4912,"num_read_kb":18971,"num_write":327,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.d","version":"108'621","reported_seq":3602,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.067232+0000","last_change":"2024-09-21T22:52:00.067893+0000","last_active":"2024-09-21T22:52:00.067232+0000","last_peered":"2024-09-21T22:52:00.067232+0000","last_clean":"2024-09-21T22:52:00.067232+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:00.067232+0000","last_undegraded":"2024-09-21T22:52:00.067232+0000","last_fullsized":"2024-09-21T22:52:00.067232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:02:57.441184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029273340000000002,"stat_sum":{"num_bytes":6135382,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":35,"num_read":5699,"num_read_kb":8778,"num_write":577,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.e","version":"108'620","reported_seq":6395,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.061958+0000","last_change":"2024-09-21T22:52:00.062428+0000","last_active":"2024-09-21T22:52:00.061958+0000","last_peered":"2024-09-21T22:52:00.061958+0000","last_clean":"2024-09-21T22:52:00.061958+0000","last_became_active":"2024-09-21T22:50:07.035768+0000","last_became_peered":"2024-09-21T22:50:07.035768+0000","last_unstale":"2024-09-21T22:52:00.061958+0000","last_undegraded":"2024-09-21T22:52:00.061958+0000","last_fullsized":"2024-09-21T22:52:00.061958+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:58.700758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169703,"stat_sum":{"num_bytes":15222617,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":34,"num_read":7666,"num_read_kb":20511,"num_write":803,"num_write_kb":9358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.f","version":"108'411","reported_seq":5136,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.064752+0000","last_change":"2024-09-21T22:52:00.066412+0000","last_active":"2024-09-21T22:52:00.064752+0000","last_peered":"2024-09-21T22:52:00.064752+0000","last_clean":"2024-09-21T22:52:00.064752+0000","last_became_active":"2024-09-21T22:50:07.035209+0000","last_became_peered":"2024-09-21T22:50:07.035209+0000","last_unstale":"2024-09-21T22:52:00.064752+0000","last_undegraded":"2024-09-21T22:52:00.064752+0000","last_fullsized":"2024-09-21T22:52:00.064752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:55:44.725451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045016190000000001,"stat_sum":{"num_bytes":26815067,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":5869,"num_read_kb":8766,"num_write":453,"num_write_kb":2859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.10","version":"110'579","reported_seq":7245,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T22:52:01.059298+0000","last_change":"2024-09-21T22:52:01.059516+0000","last_active":"2024-09-21T22:52:01.059298+0000","last_peered":"2024-09-21T22:52:01.059298+0000","last_clean":"2024-09-21T22:52:01.059298+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:01.059298+0000","last_undegraded":"2024-09-21T22:52:01.059298+0000","last_fullsized":"2024-09-21T22:52:01.059298+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:30.133916+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.014601054,"stat_sum":{"num_bytes":10208651,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":31,"num_read":8440,"num_read_kb":38719,"num_write":582,"num_write_kb":9757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"7"}]},{"pgid":"3.11","version":"105'905","reported_seq":5715,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.062638+0000","last_change":"2024-09-21T22:52:00.064252+0000","last_active":"2024-09-21T22:52:00.062638+0000","last_peered":"2024-09-21T22:52:00.062638+0000","last_clean":"2024-09-21T22:52:00.062638+0000","last_became_active":"2024-09-21T22:50:07.034427+0000","last_became_peered":"2024-09-21T22:50:07.034427+0000","last_unstale":"2024-09-21T22:52:00.062638+0000","last_undegraded":"2024-09-21T22:52:00.062638+0000","last_fullsized":"2024-09-21T22:52:00.062638+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:58.400456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027970310000000002,"stat_sum":{"num_bytes":7390237,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":28,"num_read":7278,"num_read_kb":52267,"num_write":1235,"num_write_kb":30194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.12","version":"109'512","reported_seq":8206,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.067943+0000","last_change":"2024-09-21T22:52:00.068558+0000","last_active":"2024-09-21T22:52:00.067943+0000","last_peered":"2024-09-21T22:52:00.067943+0000","last_clean":"2024-09-21T22:52:00.067943+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:00.067943+0000","last_undegraded":"2024-09-21T22:52:00.067943+0000","last_fullsized":"2024-09-21T22:52:00.067943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:52.342715+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01184909,"stat_sum":{"num_bytes":15035576,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":9593,"num_read_kb":22904,"num_write":475,"num_write_kb":7414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.13","version":"106'547","reported_seq":7596,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T22:52:00.066509+0000","last_change":"2024-09-21T22:52:00.067140+0000","last_active":"2024-09-21T22:52:00.066509+0000","last_peered":"2024-09-21T22:52:00.066509+0000","last_clean":"2024-09-21T22:52:00.066509+0000","last_became_active":"2024-09-21T22:50:07.034404+0000","last_became_peered":"2024-09-21T22:50:07.034404+0000","last_unstale":"2024-09-21T22:52:00.066509+0000","last_undegraded":"2024-09-21T22:52:00.066509+0000","last_fullsized":"2024-09-21T22:52:00.066509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:19.624998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049258740000000002,"stat_sum":{"num_bytes":24532648,"num_objects":74,"num_object_clones":21,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":7382,"num_read_kb":28275,"num_write":612,"num_write_kb":14949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"17","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":354948450,"num_objects":1914,"num_object_clones":547,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":1013,"num_read":215704,"num_read_kb":712629,"num_write":23280,"num_write_kb":331668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":43594266,"num_keys_recovered":67,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149069824,"data_stored":178500112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":19314,"ondisk_log_size":19314,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476765,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85820,"kb_used_data":31216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94286020,"statfs":{"total":96636764160,"available":96548884480,"internally_reserved":0,"allocated":31965184,"data_stored":39473332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74708,"kb_used_data":24648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94297132,"statfs":{"total":96636764160,"available":96560263168,"internally_reserved":0,"allocated":25239552,"data_stored":27754272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626451,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":4556,"kb_used_omap":4,"kb_used_meta":32251,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":4665344,"data_stored":4071641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":33025746},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111920,"kb_used_data":32156,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94259920,"statfs":{"total":96636764160,"available":96522158080,"internally_reserved":0,"allocated":32927744,"data_stored":39981676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45872,"kb_used_data":6368,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325968,"statfs":{"total":96636764160,"available":96589791232,"internally_reserved":0,"allocated":6520832,"data_stored":7668151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42792,"kb_used_data":11596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329048,"statfs":{"total":96636764160,"available":96592945152,"internally_reserved":0,"allocated":11874304,"data_stored":13231933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83840,"kb_used_data":26284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94288000,"statfs":{"total":96636764160,"available":96550912000,"internally_reserved":0,"allocated":26914816,"data_stored":29356861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056645,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86348,"kb_used_data":52632,"kb_used_omap":42,"kb_used_meta":33429,"kb_avail":94285492,"statfs":{"total":96636764160,"available":96548343808,"internally_reserved":0,"allocated":53895168,"data_stored":56338255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":34232309},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20115456,"data_stored":23261920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9236480,"data_stored":11290842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5230592,"data_stored":7066330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30171136,"data_stored":33313958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31637504,"data_stored":39379855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29638656,"data_stored":37843893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3362816,"data_stored":3463453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":19677184,"data_stored":22879861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:05.892 INFO:tasks.thrashosds.thrasher:Setting 3.19 to [0, 5, 3, 4] 2024-09-21T22:52:05.892 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.19', '0', '5', '3', '4'] 2024-09-21T22:52:05.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.19 0 5 3 4 2024-09-21T22:52:05.908 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:05.908 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:05.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:05.976 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:05.976+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:06.043 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:06.040+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:06.044 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:06.045 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:06.059 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:06.080 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:52:06.143 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:06.138+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:06.199 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.177974402000004, 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.173736833000007, 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.173541935000003, 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451867305999997, 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451825425999999, 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451789840000004, 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451757657999998, 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451725631999999, 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451655619999997, 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451591741000001, 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451538497999998, 2024-09-21T22:52:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451487188999998, 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451436364000003, 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451389204999998, 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451353669, 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:06.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:06.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451305439999999, 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:06.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:06.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451237298000002, 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:06.274 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.278 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451176162000003, 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:06.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:06.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451117994000001, 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.451061844999998, 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:06.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:06.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:06.291 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:06.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:06.292 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:06.286+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:06.393 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:06.392+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:06.415 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:06.415 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:06.415 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:06.415 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:06.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:06.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:06.516 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:06.516+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:06.588 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:06.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:06.588 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:06.588 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:06.588 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:06.594 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:06.592+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:06.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:06.617 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:06.633 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:06.651 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:06.651 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:06.668 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:06.668 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:06.689 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:06.689 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:06.695 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:06.692+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:06.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:06.710 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:06.728 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:06.728 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:06.730 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:06.730 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:06.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:06.751 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:06.797 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:06.790+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:06.859 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:06.859 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:06.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:06.898 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:06.894+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:06.987 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:06.987 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:06.999 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:06.996+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:07.002 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.19 pg_upmap_items mapping to [0->5,3->4] 2024-09-21T22:52:07.099 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:07.096+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:07.171 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.171 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.184 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:07.232 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:07.232+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:07.300 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:07.300+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:07.302 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.302 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:07.401 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:07.394+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:07.475 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.475 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:07.502 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:07.498+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:07.602 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:07.600+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:07.712 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:07.712+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:07.792 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.792 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.803 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:07.802+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:07.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:07.904 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:07.904+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:07.972 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:07.972 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:07.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:08.005 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:08.002+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:08.106 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:08.102+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:08.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:52:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:08.221 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:08.218+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:08.307 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:08.302+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:08.323 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.323 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:08.426 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:08.422+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.509 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:08.508+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:08.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:08.610 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:08.606+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:08.733 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:08.730+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:08.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:08.812 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:08.812+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:08.911 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:08.912+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:08.940 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:08.940 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:08.959 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:08.959 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:08.976 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:08.991 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:08.991 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:08.997 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:08.997 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:09.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:09.013 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:09.012+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:09.017 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:09.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:09.035 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:09.053 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:09.105 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:52:09.115 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:09.110+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.861039132999998, 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:09.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860997287000004, 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:52:09.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.153 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860951813, 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:52:09.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.158 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860916885000002, 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:52:09.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860884638999998, 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860848906999998, 2024-09-21T22:52:09.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860811730999998, 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.860777837999997, 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:09.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:09.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 50.852342493999998, 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164846 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1622016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183325760999999, 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062572554000000002, 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164846 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954226+0000", 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954228+0000", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.057e-06 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954232+0000", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954249+0000", 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6690000000000001e-05 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954262+0000", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3312999999999999e-05 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016014+0000", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061751383 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016230+0000", 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021614099999999999 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016799+0000", 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056914699999999999 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183274142000002, 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183222839000003, 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:52:09.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:52:09.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183164923, 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:52:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:52:09.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183109264000002, 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:09.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.183053115, 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.209 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:09.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:52:09.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:52:09.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.182990623000002, 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:52:09.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.182889805000002, 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:52:09.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:52:09.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.182833985999999, 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:52:09.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:52:09.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:52:09.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:52:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.18278128, 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:52:09.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:52:09.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:52:09.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:52:09.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.634687607, 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:52:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:52:09.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:52:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:52:09.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.237 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:09.239 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:09.236+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:09.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:09.339 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:09.336+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:09.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:09.420 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:09.421 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:09.421 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:09.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:09.481 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:09.478+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:09.542 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:09.540+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:09.563 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:09.563 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:09.564 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:09.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:09.665 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:09.662+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:09.743 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:09.738+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:09.747 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 136.81096481700001, 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:52:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:52:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:52:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265892957999995, 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:52:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265829507999996, 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:52:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265766330000005, 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:52:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:52:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265709990000005, 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:52:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265668247999997, 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:52:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265607646999996, 2024-09-21T22:52:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:52:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265527211000006, 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:52:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265451100999996, 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:52:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:52:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265364027999993, 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265320540000005, 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:52:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265286902, 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:52:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:52:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265253135999998, 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:52:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:52:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265221851999996, 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:52:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:52:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265176496999999, 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:52:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:52:09.826 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265145000999993, 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:52:09.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:52:09.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.265110219999997, 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:52:09.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:52:09.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.247103644999996, 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:52:09.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.924523495000003, 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:09.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:52:09.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:09.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.223085306000002, 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:09.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:09.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:52:09.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:52:09.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:52:09.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:09.856 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:09.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:09.858 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:09.854+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:09.958 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:09.954+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:10.028 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:10.028 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:10.028 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:10.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:10.085 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:10.082+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:10.164 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:10.160+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:10.229 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:10.229 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:10.229 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:10.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:10.261 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:10.260+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:10.362 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:10.360+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.352231724000006, 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:10.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:10.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:10.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.347994154999995, 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:10.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:10.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.347799257000005, 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:10.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:10.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:10.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:10.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.395 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.626124627999999, 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:10.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.626082748000002, 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:10.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.404 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.626047161999999, 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:10.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:10.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.626014980000001, 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:10.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625982954000001, 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625912941999999, 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625849062999997, 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:10.426 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.62579582, 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:10.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625744511000001, 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625693685999998, 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:10.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625646527000001, 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:10.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:10.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:10.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625610991000002, 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:10.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625562762000001, 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:10.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625494619999998, 2024-09-21T22:52:10.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:10.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:10.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:10.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625433483999998, 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:10.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625375316000003, 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:10.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.625319167000001, 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:10.466 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:10.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:10.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.470 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:10.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:10.471 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:10.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:10.472 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:10.466+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:10.573 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:10.566+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:10.602 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:10.602 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:10.602 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:10.602 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:10.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:10.673 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:10.672+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:10.738 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:10.738 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:10.738 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:10.738 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:10.738 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:10.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:10.774 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:10.770+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:10.874 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:10.872+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.413446183000001, 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:10.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:10.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:10.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:10.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:10.927 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.387594052000001, 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:10.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.302216437999999, 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:10.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:10.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.938 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.299523999000002, 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.878785417, 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:10.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3017 3.c e105/90 3:31662d49:::rbd_data.111e6870f364.00000000000002bb:1b v 105'300, mlcod=100'294)", 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:10.947 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.868598167, 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005811498, 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:10.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049528+0000", 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049538+0000", 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0713000000000001e-05 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049559+0000", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1146e-05 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049575+0000", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5376999999999999e-05 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051168+0000", 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001593325 2024-09-21T22:52:10.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051221+0000", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3066000000000003e-05 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055238+0000", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040165380000000001 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055338+0000", 2024-09-21T22:52:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100183 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.867941445, 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:10.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.866435833000001, 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.956 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:10.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:173694 3.1f 3:fedae506:::rbd_object_map.111e6870f364:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.865303008, 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059303280000000003, 2024-09-21T22:52:10.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 173694 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052825+0000", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.123e-06 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052839+0000", 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4059999999999999e-05 2024-09-21T22:52:10.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052871+0000", 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1717000000000001e-05 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052897+0000", 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6979999999999999e-05 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054139+0000", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001241881 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054646+0000", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050662199999999998 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055467+0000", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082064499999999995 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058198+0000", 2024-09-21T22:52:10.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027309040000000001 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058349+0000", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015121500000000001 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058661+0000", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031251800000000002 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058752+0000", 2024-09-21T22:52:10.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0662999999999996e-05 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.853731957000001, 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:10.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:10.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.85445726, 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.970 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:10.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.853054811, 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:10.974 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8714434460000007, 2024-09-21T22:52:10.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8472689839999994, 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174548 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8461551049999994, 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005864953, 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174548 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071972+0000", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6929999999999999e-06 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071987+0000", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.431e-05 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072021+0000", 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4607000000000003e-05 2024-09-21T22:52:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072043+0000", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1756999999999999e-05 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073484+0000", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001440906 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073732+0000", 2024-09-21T22:52:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000248571 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077250+0000", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035174809999999998 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077469+0000", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021931300000000001 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077834+0000", 2024-09-21T22:52:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036531500000000001 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3061 3.c e110/90 3:36f8a985:::rbd_data.111ec49ace6d.0000000000000002:16 v 110'320, mlcod=107'314)", 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.8459872789999991, 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056352640000000001, 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072147+0000", 2024-09-21T22:52:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4830000000000005e-06 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072160+0000", 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.29e-05 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072185+0000", 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5092999999999998e-05 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072201+0000", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6082999999999998e-05 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073185+0000", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000984113 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073230+0000", 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4620000000000003e-05 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077683+0000", 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044532210000000003 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077773+0000", 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9751000000000002e-05 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175151 3.1f 3:fc822b53:::rbd_data.111e5a344aa.000000000000001a:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T22:52:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7232066010000002, 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058184259999999998, 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175151 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194919+0000", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2099999999999995e-07 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194924+0000", 2024-09-21T22:52:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194938+0000", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4273e-05 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194946+0000", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2130000000000008e-06 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195083+0000", 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013696199999999999 2024-09-21T22:52:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195288+0000", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020458 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195451+0000", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000163499 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.199272+0000", 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038211360000000002 2024-09-21T22:52:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200504+0000", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001231851 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200652+0000", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014794799999999999 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200736+0000", 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.441e-05 2024-09-21T22:52:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175586 3.18 3:18265a29:::rbd_header.111e4aac08c7:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.7997006, 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058485500000000001, 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175586 2024-09-21T22:52:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118426+0000", 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-21T22:52:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118433+0000", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7610000000000002e-06 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118449+0000", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6535999999999999e-05 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118461+0000", 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1912999999999999e-05 2024-09-21T22:52:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119273+0000", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081154999999999997 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119383+0000", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109811 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119873+0000", 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049069200000000004 2024-09-21T22:52:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.122242+0000", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023691469999999998 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124142+0000", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018992099999999999 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124213+0000", 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1901000000000002e-05 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124273+0000", 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9086000000000003e-05 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.6484864479999999, 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.77891202800000003, 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:11.018 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:11.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:11.020 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:11.016+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:11.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:11.121 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:11.114+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:11.209 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:11.209 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:11.222 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:11.218+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:11.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:11.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:52:11.260 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:11.279 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:11.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:11.313 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:11.323 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:11.318+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:11.334 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:11.334 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:11.348 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:11.348 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:11.355 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:11.363 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:11.376 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:11.377 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:11.398 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:11.398 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:11.424 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:11.418+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:11.493 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:11.493 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:11.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:11.524 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:11.522+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:11.625 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:11.622+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:11.625 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:11.626 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:11.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:11.726 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:11.724+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:11.761 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:11.761 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:11.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:11.827 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:11.822+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:11.927 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:11.924+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:11.940 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:11.940 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:11.954 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:12.040 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:12.040+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:12.118 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 7, 6, 3] 2024-09-21T22:52:12.118 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:52:12.118 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T22:52:12.118 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.889059 2024-09-21T22:52:12.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.8890589406360409 2024-09-21T22:52:12.119 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:12.119 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:52:12.130 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:12.122+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:12.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:12.230 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:12.226+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:12.328 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:12.329 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:12.331 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:12.328+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:12.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:12.432 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:12.426+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:12.504 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:12.504 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:12.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:12.565 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:12.562+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:12.633 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:12.630+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:12.651 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:12.652 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:12.666 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:12.734 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:12.732+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:12.831 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:12.831 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:12.835 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:12.830+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:12.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:12.936 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:12.936+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:13.010 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:13.010 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:13.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:13.037 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:13.036+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.4 primary-affinity to 0.889059 (8582652) 2024-09-21T22:52:13.138 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:13.136+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:13.155 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:13.155 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:13.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:13.239 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:13.236+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:13.306 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:13.306 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:13.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:13.340 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:13.340+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:13.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:13.440 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:13.440 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:13.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:13.464 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:13.464+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:13.541 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:13.538+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:13.572 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:13.572 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:13.578 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:13.578 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:13.588 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:13.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:13.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:13.621 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:13.622 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:13.642 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:13.640+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:13.642 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:13.642 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:13.662 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:13.662 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:13.681 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:13.681 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:13.700 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:13.760 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:13.760+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:13.763 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:13.763 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:13.763 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:13.764 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:13.764 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:13.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:13.843 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:13.838+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:52:13.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 140.964433294, 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:13.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:52:13.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:52:13.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419361434999999, 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419297985, 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419234806999995, 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:52:13.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419178466999995, 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419136725000001, 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:52:13.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.419076124, 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418995687999995, 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:13.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:52:13.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418919578000001, 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:52:13.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418832504999997, 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418789016999995, 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:52:13.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418755379000004, 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:52:13.950 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418721613000002, 2024-09-21T22:52:13.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418690329, 2024-09-21T22:52:13.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:13.962 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418644974000003, 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.966 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418613477999997, 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:52:13.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:52:13.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.418578697000001, 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:52:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.400572122, 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:52:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.077991972, 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:52:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:52:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.376553782999999, 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:13.990 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:13.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:13.992 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:13.988+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:14.093 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:14.086+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:14.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:52:14.166 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:14.166 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:14.166 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:14.166 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:14.183 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:14.200 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:14.200+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:14.294 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:14.292+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:14.359 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:14.359 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:14.359 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:14.359 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:14.360 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:14.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:14.395 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:14.392+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:14.496 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:14.496+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 98.476311350000003, 2024-09-21T22:52:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 98.472073781000006, 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 98.471878883000002, 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.750204254000003, 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.750162373999999, 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.750126788000003, 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.750094605999998, 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.750062579999998, 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749992568000003, 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749928689000001, 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749875445999997, 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749824136999997, 2024-09-21T22:52:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749773312000002, 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749726152999997, 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749690616999999, 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749642387999998, 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749574246000002, 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749513110000002, 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749454942, 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.749398792999997, 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:14.581 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:14.582 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:14.628 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:14.628+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:14.697 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:14.694+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:14.708 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:14.708 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:14.709 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:14.709 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:14.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:14.798 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:14.794+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:14.889 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:14.889 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:14.889 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:14.889 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:14.889 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:14.900 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:14.900+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:14.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:15.001 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:15.000+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.565630773000002, 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:15.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.539778642000002, 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.454401027999999, 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:15.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:15.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:15.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.451708588999999, 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.091 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.030970007000001, 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3017 3.c e105/90 3:31662d49:::rbd_data.111e6870f364.00000000000002bb:1b v 105'300, mlcod=100'294)", 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.020782756999999, 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005811498, 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049528+0000", 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049538+0000", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0713000000000001e-05 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049559+0000", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1146e-05 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049575+0000", 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5376999999999999e-05 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051168+0000", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001593325 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051221+0000", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3066000000000003e-05 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055238+0000", 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040165380000000001 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055338+0000", 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100183 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.020126035000001, 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:15.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:15.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.018620423000002, 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:15.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.110 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:173694 3.1f 3:fedae506:::rbd_object_map.111e6870f364:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.017487597999999, 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059303280000000003, 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 173694 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052825+0000", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.123e-06 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052839+0000", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4059999999999999e-05 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052871+0000", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1717000000000001e-05 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052897+0000", 2024-09-21T22:52:15.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6979999999999999e-05 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054139+0000", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001241881 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054646+0000", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050662199999999998 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055467+0000", 2024-09-21T22:52:15.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082064499999999995 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058198+0000", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027309040000000001 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058349+0000", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015121500000000001 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058661+0000", 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031251800000000002 2024-09-21T22:52:15.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058752+0000", 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0662999999999996e-05 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.005916547000002, 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:15.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:15.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.006641849999999, 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:15.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.123 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.005239401000001, 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:15.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.023628036, 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:15.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.999453574, 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:15.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:15.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.138 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174548 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.998339695, 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005864953, 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174548 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071972+0000", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6929999999999999e-06 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071987+0000", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.431e-05 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072021+0000", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4607000000000003e-05 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072043+0000", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1756999999999999e-05 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073484+0000", 2024-09-21T22:52:15.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001440906 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073732+0000", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000248571 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077250+0000", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035174809999999998 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077469+0000", 2024-09-21T22:52:15.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021931300000000001 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077834+0000", 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036531500000000001 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3061 3.c e110/90 3:36f8a985:::rbd_data.111ec49ace6d.0000000000000002:16 v 110'320, mlcod=107'314)", 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.998171869, 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056352640000000001, 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072147+0000", 2024-09-21T22:52:15.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4830000000000005e-06 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072160+0000", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.29e-05 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072185+0000", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5092999999999998e-05 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072201+0000", 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6082999999999998e-05 2024-09-21T22:52:15.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073185+0000", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000984113 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073230+0000", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4620000000000003e-05 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077683+0000", 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044532210000000003 2024-09-21T22:52:15.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077773+0000", 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9751000000000002e-05 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175151 3.1f 3:fc822b53:::rbd_data.111e5a344aa.000000000000001a:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8753911910000003, 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058184259999999998, 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175151 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194919+0000", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2099999999999995e-07 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194924+0000", 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194938+0000", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4273e-05 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194946+0000", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2130000000000008e-06 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195083+0000", 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013696199999999999 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195288+0000", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020458 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195451+0000", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000163499 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.199272+0000", 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038211360000000002 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200504+0000", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001231851 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200652+0000", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014794799999999999 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200736+0000", 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.441e-05 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175586 3.18 3:18265a29:::rbd_header.111e4aac08c7:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.9518851900000005, 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058485500000000001, 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175586 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118426+0000", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118433+0000", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7610000000000002e-06 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118449+0000", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6535999999999999e-05 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118461+0000", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1912999999999999e-05 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119273+0000", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081154999999999997 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119383+0000", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109811 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119873+0000", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049069200000000004 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.122242+0000", 2024-09-21T22:52:15.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023691469999999998 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124142+0000", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018992099999999999 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124213+0000", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1901000000000002e-05 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124273+0000", 2024-09-21T22:52:15.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9086000000000003e-05 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.8006710379999999, 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:15.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:15.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9310966179999998, 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:15.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:15.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.167 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:15.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:52:15.169 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:15.168+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:15.269 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:15.268+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:15.340 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:15.340 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:15.340 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:15.340 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:15.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:15.400 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:15.400+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:15.471 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:15.468+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:15.522 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:15.522 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:15.522 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:15.522 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:15.522 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:15.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:15.572 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:15.566+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:15.673 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:15.666+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 57.417644887000002, 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.581385273999999, 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:52:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467708260000002, 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467559698000002, 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467489317999998, 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:52:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467459105000003, 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467422845000002, 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:52:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:52:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467391151999998, 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467358918999999, 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:52:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467328690000002, 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.467292096000001, 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:52:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759915642000003, 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759805202000003, 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:15.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:52:15.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:52:15.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759728346000003, 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759660513999997, 2024-09-21T22:52:15.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759590244000002, 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:52:15.765 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:52:15.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759505591, 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:52:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759445215, 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.773 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:52:15.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759386671999998, 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:52:15.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:52:15.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.759300729000003, 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:52:15.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:52:15.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:15.784 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:15.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:15.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:15.786 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:15.782+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:15.886 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:15.886+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:15.948 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:15.948 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:15.948 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:15.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:15.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:15.981 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:15.987 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:15.984+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:15.997 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:16.014 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:16.014 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:16.032 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:16.061 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:16.061 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:16.084 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:16.088 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:16.082+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:16.090 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.090 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.107 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:16.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:16.129 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:16.196 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:16.196+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:16.251 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.251 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.265 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:16.289 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:16.286+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:16.391 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:16.386+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:16.427 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.427 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:16.491 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:16.486+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:16.565 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.565 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:16.592 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:16.586+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:16.693 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:16.686+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:16.794 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:16.790+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:16.846 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:16.847 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:16.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:16.895 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:16.892+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:16.996 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:16.990+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:17.024 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.025 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:17.125 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:17.122+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:17.170 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T22:52:17.198 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:17.194+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.217 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:17.299 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:17.296+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:17.400 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:17.400+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:17.402 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.402 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:17.501 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:17.498+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:17.540 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.540 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:17.602 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:17.600+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:17.703 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:17.698+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:17.719 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.719 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:17.804 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:17.804+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:17.903 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:17.903 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:17.905 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:17.902+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:17.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:18.006 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:18.002+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:18.086 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:18.087 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:18.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:18.146 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:18.142+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:18.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:18.161 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 7, 6, 3] 2024-09-21T22:52:18.161 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:52:18.161 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T22:52:18.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-21T22:52:18.163 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-21T22:52:18.251 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:18.252+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:18.301 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:18.308 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:18.308+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:18.324 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:18.324 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:18.342 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:18.342 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:18.359 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:18.359 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:18.375 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:18.396 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:18.409 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:18.408+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:18.413 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:18.413 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:18.422 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:18.422 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:18.433 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:18.433 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:18.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:18.452 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:18.500 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-21T22:52:18.500+0000 7fe4f0d43640 -1 mon.a@0(leader).osd e127 definitely_dead 0 2024-09-21T22:52:18.510 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:18.506+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:18.560 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:18.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:18.560 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:18.560 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:18.560 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:18.574 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:18.617 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:18.614+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:18.711 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:18.712+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 102.720394353, 2024-09-21T22:52:18.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 102.71615678400001, 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 102.715961886, 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994287257000003, 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994245376999999, 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994209791000003, 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994177608999998, 2024-09-21T22:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994145582999998, 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994075571000003, 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.994011692000001, 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993958448999997, 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993907139999997, 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993856315000002, 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993809155999998, 2024-09-21T22:52:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993773619999999, 2024-09-21T22:52:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993725390999998, 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993657249000002, 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993596113000002, 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993537945, 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.993481795999998, 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:18.834 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:18.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:18.835 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:18.832+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:18.936 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:18.936+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:18.965 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:18.965 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:18.965 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:18.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:19.037 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:19.034+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:19.138 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:19.134+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:19.147 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:19.147 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:19.147 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:19.147 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:19.147 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:19.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:19.173 INFO:teuthology.orchestra.run.smithi094.stderr:marked out osd.4. 2024-09-21T22:52:19.239 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:19.234+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:19.288 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.782725607000003, 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:19.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:19.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:19.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:19.296 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.756873476000003, 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.671495862, 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:19.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:19.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:19.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.668803423, 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.248064841000001, 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:19.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.315 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3017 3.c e105/90 3:31662d49:::rbd_data.111e6870f364.00000000000002bb:1b v 105'300, mlcod=100'294)", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.237877591, 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005811498, 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:19.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049528+0000", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049538+0000", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0713000000000001e-05 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049559+0000", 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1146e-05 2024-09-21T22:52:19.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049575+0000", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5376999999999999e-05 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051168+0000", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001593325 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051221+0000", 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3066000000000003e-05 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055238+0000", 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040165380000000001 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055338+0000", 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100183 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.319 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.237220869000001, 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:19.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:19.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.235715256999999, 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:173694 3.1f 3:fedae506:::rbd_object_map.111e6870f364:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.234582432, 2024-09-21T22:52:19.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059303280000000003, 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 173694 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052825+0000", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.123e-06 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052839+0000", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4059999999999999e-05 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052871+0000", 2024-09-21T22:52:19.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1717000000000001e-05 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052897+0000", 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6979999999999999e-05 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054139+0000", 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001241881 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054646+0000", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050662199999999998 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055467+0000", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082064499999999995 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058198+0000", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027309040000000001 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058349+0000", 2024-09-21T22:52:19.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015121500000000001 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058661+0000", 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031251800000000002 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058752+0000", 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0662999999999996e-05 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.223011380999999, 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:19.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:19.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:19.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.223736683999999, 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.337 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:19.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:19.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.222334235000002, 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:19.341 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.240722869999999, 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:19.345 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:19.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.216548408000001, 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:19.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:19.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174548 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.215434528999999, 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005864953, 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174548 2024-09-21T22:52:19.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071972+0000", 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6929999999999999e-06 2024-09-21T22:52:19.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071987+0000", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.431e-05 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072021+0000", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4607000000000003e-05 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072043+0000", 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1756999999999999e-05 2024-09-21T22:52:19.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073484+0000", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001440906 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073732+0000", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000248571 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077250+0000", 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035174809999999998 2024-09-21T22:52:19.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077469+0000", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021931300000000001 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077834+0000", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036531500000000001 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3061 3.c e110/90 3:36f8a985:::rbd_data.111ec49ace6d.0000000000000002:16 v 110'320, mlcod=107'314)", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:19.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.215266703000001, 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056352640000000001, 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072137+0000", 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072147+0000", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4830000000000005e-06 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072160+0000", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.29e-05 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072185+0000", 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5092999999999998e-05 2024-09-21T22:52:19.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072201+0000", 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6082999999999998e-05 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073185+0000", 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000984113 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073230+0000", 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4620000000000003e-05 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077683+0000", 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044532210000000003 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077773+0000", 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9751000000000002e-05 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175151 3.1f 3:fc822b53:::rbd_data.111e5a344aa.000000000000001a:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.092486024999999, 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058184259999999998, 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175151 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.365 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194919+0000", 2024-09-21T22:52:19.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2099999999999995e-07 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194924+0000", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194938+0000", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4273e-05 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194946+0000", 2024-09-21T22:52:19.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2130000000000008e-06 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195083+0000", 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013696199999999999 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195288+0000", 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020458 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:19.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195451+0000", 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000163499 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.199272+0000", 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038211360000000002 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200504+0000", 2024-09-21T22:52:19.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001231851 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200652+0000", 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014794799999999999 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200736+0000", 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.441e-05 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.370 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175586 3.18 3:18265a29:::rbd_header.111e4aac08c7:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.168980024, 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058485500000000001, 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175586 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118426+0000", 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118433+0000", 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7610000000000002e-06 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118449+0000", 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6535999999999999e-05 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118461+0000", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1912999999999999e-05 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119273+0000", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081154999999999997 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119383+0000", 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109811 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119873+0000", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049069200000000004 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.122242+0000", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023691469999999998 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124142+0000", 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018992099999999999 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124213+0000", 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1901000000000002e-05 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124273+0000", 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9086000000000003e-05 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.376 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.017765872, 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1481914520000007, 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:19.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:19.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:19.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.386 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:19.387 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:19.384+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:19.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:19.488 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:19.488+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:19.559 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:19.559 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:19.559 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:19.559 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:19.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:19.620 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:19.620+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:19.689 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:19.686+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:19.736 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:19.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:19.736 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:19.736 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:19.736 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:19.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:19.796 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:19.796+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:19.870 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:19.870 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:19.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.582655189, 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:52:19.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.746395575999998, 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.882 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632718562000001, 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:52:19.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632570000000001, 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.887 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:52:19.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632499619999997, 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:52:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:52:19.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632469407000002, 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632433147, 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:52:19.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:52:19.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632401454000004, 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632369220999998, 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:52:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632338992000001, 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.632302398, 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:52:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924925944000002, 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:52:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:52:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:52:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924815504000001, 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924738648000002, 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:19.931 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924670816000003, 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:52:19.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924600546000001, 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:52:19.939 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924515892999999, 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924455516999998, 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.944 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:19.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:19.948 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924396973999997, 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:19.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:52:19.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.924311031000002, 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:52:19.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:52:19.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:52:19.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:52:19.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:19.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:19.958 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:19.956+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:20.058 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:20.058+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:20.118 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:20.118 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:20.118 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:20.119 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:20.132 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:20.159 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:20.158+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:20.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T22:52:20.246 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:20.246 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:20.246 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:20.246 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:20.246 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:20.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:20.260 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:20.258+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:20.360 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:20.354+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:20.426 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:20.426 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:20.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.336661225, 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:52:20.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:20.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.936461359999996, 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.431 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:52:20.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934947100000002, 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:52:20.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934893798000004, 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.440 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934838749999997, 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.448 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934769566, 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:52:20.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:52:20.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:52:20.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934706998999999, 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:52:20.453 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:52:20.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:52:20.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934657604999998, 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.458 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:52:20.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:52:20.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934153821999999, 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.934086242999996, 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:20.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933537000000001, 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:20.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933459518999996, 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:52:20.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933295388000005, 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:52:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933234408999994, 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:52:20.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:52:20.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933190796999995, 2024-09-21T22:52:20.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:20.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933146315000002, 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:52:20.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933101147000002, 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:52:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.501 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.933058849999995, 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 90.447626260999996, 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 57.096131786999997, 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:20.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:20.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:20.519 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:20.516+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:20.618 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:20.614+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:20.679 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:20.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:20.680 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:20.680 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:20.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:20.707 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:20.719 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:20.716+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:20.723 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:20.724 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:20.739 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:20.739 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:20.756 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:20.756 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:20.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:20.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:20.795 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:20.796 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:20.807 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:20.807 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:20.815 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:20.815 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:20.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:20.820 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:20.814+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:20.836 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:20.920 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:20.918+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:20.946 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:20.946 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:20.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:21.022 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:21.020+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:21.051 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-21T22:52:21.122 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:21.118+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:21.128 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:21.128 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:21.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:21.223 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:21.224+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:21.319 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:21.319 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:21.324 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:21.318+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:21.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:21.425 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:21.424+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:21.521 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:21.521 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:21.526 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:21.524+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:21.540 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:21.627 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:21.622+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:21.678 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:21.678 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:21.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:21.741 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:21.738+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:21.827 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:21.822+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:21.867 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:21.867 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:21.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:21.973 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:21.970+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:22.029 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:22.026+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:22.069 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.069 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:22.130 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:22.128+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:22.230 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:22.226+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:22.278 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.278 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:22.337 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:22.334+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:22.432 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:22.432+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:22.479 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.479 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:22.532 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:22.532+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:22.622 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.622 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.634 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:22.632+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:22.638 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:22.735 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:22.730+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:22.764 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.765 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:22.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:22.869 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:22.866+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:22.937 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:22.934+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:22.959 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:22.959 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:22.976 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:23.008 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:23.008 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:23.024 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:23.024 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:23.039 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:23.036+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:23.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:23.041 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:23.058 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:23.059 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:23.076 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:23.077 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:23.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:52:23.093 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:23.112 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:23.129 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:23.140 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:23.140+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:23.158 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:23.158 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:23.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:23.158 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.13770542100001, 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.159 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:23.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:23.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:23.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:23.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.133467852, 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.13327295400001, 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411598325, 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411556445000002, 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411520858999999, 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:23.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411488677000001, 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:23.190 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:23.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411456651000002, 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:23.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:23.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411386639, 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:23.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411322759999997, 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411269517000001, 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411218208000001, 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:23.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:23.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411167382999999, 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411120224000001, 2024-09-21T22:52:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411084688000003, 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.411036459000002, 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.410968316999998, 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.410907180999999, 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.410849013000004, 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.410792864000001, 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:23.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:23.248 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:23.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:23.249 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:23.248+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:23.349 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:23.346+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:23.362 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:23.363 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:23.363 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:23.363 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:23.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:23.450 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:23.446+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:23.506 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:23.506 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:23.506 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:23.506 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:23.506 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:23.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:23.551 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:23.546+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:23.641 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:23.641 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.135371859000003, 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:23.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:23.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:23.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:23.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:23.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.109519728000002, 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:23.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.024142114, 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:23.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:23.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.021449675, 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:23.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:23.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:23.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.600711093000001, 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:23.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:23.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:23.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3017 3.c e105/90 3:31662d49:::rbd_data.111e6870f364.00000000000002bb:1b v 105'300, mlcod=100'294)", 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.590523843, 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005811498, 2024-09-21T22:52:23.667 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049526+0000", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049528+0000", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049538+0000", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0713000000000001e-05 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049559+0000", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1146e-05 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049575+0000", 2024-09-21T22:52:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5376999999999999e-05 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051168+0000", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001593325 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051221+0000", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3066000000000003e-05 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055238+0000", 2024-09-21T22:52:23.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040165380000000001 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055338+0000", 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100183 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.589867121000001, 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:23.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:23.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:23.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.588361508999999, 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:23.675 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:173694 3.1f 3:fedae506:::rbd_object_map.111e6870f364:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.587228683999999, 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059303280000000003, 2024-09-21T22:52:23.679 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 173694 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052822+0000", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052825+0000", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.123e-06 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052839+0000", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4059999999999999e-05 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052871+0000", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1717000000000001e-05 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.052897+0000", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6979999999999999e-05 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054139+0000", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001241881 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054646+0000", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050662199999999998 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.055467+0000", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082064499999999995 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058198+0000", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027309040000000001 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058349+0000", 2024-09-21T22:52:23.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015121500000000001 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058661+0000", 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031251800000000002 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058752+0000", 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0662999999999996e-05 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.684 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.575657632999999, 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:23.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:23.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.688 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.576382936000002, 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:23.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.574980487000001, 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:23.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.593369121999999, 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:23.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.569194660000001, 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:23.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:23.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174548 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.568080780999999, 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005864953, 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174548 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071970+0000", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071972+0000", 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6929999999999999e-06 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071987+0000", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.431e-05 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072021+0000", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4607000000000003e-05 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072043+0000", 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1756999999999999e-05 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073484+0000", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001440906 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.073732+0000", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000248571 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077250+0000", 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035174809999999998 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077469+0000", 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021931300000000001 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077834+0000", 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036531500000000001 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175151 3.1f 3:fc822b53:::rbd_data.111e5a344aa.000000000000001a:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T22:52:23.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.445132276999999, 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058184259999999998, 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175151 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194918+0000", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194919+0000", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2099999999999995e-07 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194924+0000", 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-21T22:52:23.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194938+0000", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4273e-05 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.194946+0000", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2130000000000008e-06 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195083+0000", 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013696199999999999 2024-09-21T22:52:23.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195288+0000", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020458 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.195451+0000", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000163499 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.199272+0000", 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038211360000000002 2024-09-21T22:52:23.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200504+0000", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001231851 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200652+0000", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014794799999999999 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:06.200736+0000", 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.441e-05 2024-09-21T22:52:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175586 3.18 3:18265a29:::rbd_header.111e4aac08c7:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.521626275999999, 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058485500000000001, 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175586 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118424+0000", 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118426+0000", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118433+0000", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7610000000000002e-06 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118449+0000", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6535999999999999e-05 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.118461+0000", 2024-09-21T22:52:23.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1912999999999999e-05 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119273+0000", 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081154999999999997 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119383+0000", 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000109811 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:23.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.119873+0000", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049069200000000004 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.122242+0000", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023691469999999998 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124142+0000", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018992099999999999 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124213+0000", 2024-09-21T22:52:23.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1901000000000002e-05 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.124273+0000", 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9086000000000003e-05 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.370412124, 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:23.723 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:23.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.500837704, 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:23.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.732 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177349 3.7 3:e2fc42f5:::rbd_header.111e7572a6e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4653032489999998, 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018545189, 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177349 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174749+0000", 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.838e-06 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174756+0000", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5519999999999999e-06 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174771+0000", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4527e-05 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174781+0000", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0481e-05 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174878+0000", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6747000000000001e-05 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175063+0000", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000184527 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175876+0000", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081354400000000003 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.177282+0000", 2024-09-21T22:52:23.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405852 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193173+0000", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015890609999999999 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193241+0000", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8131000000000002e-05 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193292+0000", 2024-09-21T22:52:23.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1379999999999999e-05 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:23.738 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:23.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:23.739 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:23.734+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:23.840 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:23.840+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:23.867 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:23.867 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:23.867 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:23.867 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:23.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:23.941 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:23.938+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:24.042 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:24.040+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:24.045 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.046 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:24.046 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:24.046 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:24.046 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:24.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:24.142 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:24.138+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:24.204 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 7, 6, 3] 2024-09-21T22:52:24.204 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T22:52:24.204 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T22:52:24.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-21T22:52:24.205 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.943349445999999, 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:52:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:52:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:52:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.107089833000003, 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:52:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:52:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:52:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993412819, 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:52:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:52:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993264257, 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:52:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:52:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:52:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993193877000003, 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:52:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993163664000001, 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:52:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:52:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993127403999999, 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993095711000002, 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993063478000003, 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.993033249, 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.992996654999999, 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:52:24.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285620201, 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:24.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285509761, 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:52:24.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.283 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285432905, 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:24.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:52:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:52:24.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285365073000001, 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285294802999999, 2024-09-21T22:52:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:52:24.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285210149999997, 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:52:24.295 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:52:24.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285149773999997, 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:24.303 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285091231000003, 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:52:24.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.285005288000001, 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.312 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:24.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:24.313 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:24.306+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:24.414 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:24.410+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:24.474 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.475 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:24.475 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:24.475 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:24.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:24.515 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:24.514+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:24.527 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-21T22:52:24.524+0000 7fe4f0d43640 -1 mon.a@0(leader).osd e133 definitely_dead 0 2024-09-21T22:52:24.607 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.607 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:24.607 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:24.607 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:24.607 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:24.615 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:24.616+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:24.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:24.716 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:24.710+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:34244 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 99.699938916999997, 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040227549000000001, 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 34244 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088180+0000", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088182+0000", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.086e-06 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088188+0000", 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.302e-06 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.088193+0000", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5009999999999998e-06 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114551+0000", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.026357823999999998 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.114618+0000", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7189999999999999e-05 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:45.128407+0000", 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013789254000000001 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.299739052000007, 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.795 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.298224791999999, 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.298171490000001, 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.298116441999994, 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:52:24.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.298047257999997, 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.297984690999996, 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.815 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.297935296999995, 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.297431513999996, 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:52:24.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:52:24.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.297363935000007, 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:52:24.827 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:52:24.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296814691999998, 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:52:24.831 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:52:24.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296737211000007, 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:52:24.835 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:52:24.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:52:24.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296573080000002, 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296512101000005, 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296468489000006, 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296424006999999, 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:52:24.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296378838999999, 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.296336542000006, 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:52:24.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:52:24.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.862 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 94.810903952999993, 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.863 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:24.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.459409479000001, 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:24.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:24.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:52:24.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:24.872 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:24.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:24.874 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:24.870+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:24.975 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:24.972+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:24.997 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:24.997 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:24.997 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:24.997 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:25.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:25.100 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:25.100+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:25.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T22:52:25.176 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:25.176 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:25.176 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:25.174+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:25.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:marked out osd.2. 2024-09-21T22:52:25.277 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:25.276+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:25.309 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:25.326 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:25.342 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:25.359 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:25.359 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:25.360 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:25.360 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:25.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:25.380 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:25.381 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:25.381 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:25.380+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:25.399 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:25.417 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:25.417 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:25.435 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:25.478 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:25.474+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:25.506 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:25.506 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:25.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:25.605 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:25.602+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:25.680 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:25.680+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:25.685 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:25.685 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:25.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:25.789 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:25.786+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:25.867 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:25.867 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:25.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:25.925 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:25.922+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:25.982 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:25.978+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:26.053 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.053 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:26.117 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:26.114+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:26.184 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:26.178+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:26.244 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.244 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:26.284 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:26.284+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:26.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:52:26.385 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:26.382+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:26.420 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.420 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:26.521 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:26.518+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:26.588 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:26.588+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:26.600 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.601 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:26.701 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:26.698+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.789 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:26.786+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:26.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:26.890 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-21T22:52:26.888+0000 7f3357a5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26520) UID: 0 2024-09-21T22:52:26.960 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:26.960 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:26.974 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:27.021 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-21T22:52:27.018+0000 7ffa1e0d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35357) UID: 0 2024-09-21T22:52:27.046 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:52:27.107 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:27.107 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:27.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:27.124 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-21T22:52:27.124+0000 7fb1fb30f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 39037) UID: 0 2024-09-21T22:52:27.209 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:27.206+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:27.290 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:27.290 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:27.293 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:27.290+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:27.305 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:27.394 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:27.392+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:27.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T22:52:27.470 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:27.470 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:27.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:27.494 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-21T22:52:27.490+0000 7fc6557e1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26590) UID: 0 2024-09-21T22:52:27.596 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-21T22:52:27.590+0000 7f28afd78640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26595) UID: 0 2024-09-21T22:52:27.606 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-21T22:52:27.606 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T22:52:27.624 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-21T22:52:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T22:52:27.624 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-21T22:52:27.624 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T22:52:27.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:27.640 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-21T22:52:27.641 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T22:52:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-21T22:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T22:52:27.683 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-21T22:52:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T22:52:27.696 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:27.696+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:27.707 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-21T22:52:27.707 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T22:52:27.732 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-21T22:52:27.732 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T22:52:27.753 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-21T22:52:27.754 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T22:52:27.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:27.772 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:27.772 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:27.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:27.832 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:27.832+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:27.898 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:27.894+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 45.447643984000003, 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.421791853000002, 2024-09-21T22:52:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.336414239, 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.333721799999999, 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.912983218000001, 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.902139246000001, 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.900633633999998, 2024-09-21T22:52:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.887929757999999, 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.888655061000001, 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.887252612000001, 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:27.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.905641246999998, 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:28.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:28.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.881466785000001, 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.682684249000001, 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.813109828999998, 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:28.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177349 3.7 3:e2fc42f5:::rbd_header.111e7572a6e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.7775753739999995, 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018545189, 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177349 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174749+0000", 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.838e-06 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174756+0000", 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5519999999999999e-06 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174771+0000", 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4527e-05 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174781+0000", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0481e-05 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174878+0000", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6747000000000001e-05 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175063+0000", 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000184527 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175876+0000", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081354400000000003 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.177282+0000", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405852 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193173+0000", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015890609999999999 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193241+0000", 2024-09-21T22:52:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8131000000000002e-05 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193292+0000", 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1379999999999999e-05 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3809 3.15 e134/90 3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:4 v 134'494, mlcod=128'490)", 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.7158603690000001, 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0066034850000000001, 2024-09-21T22:52:28.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.65e-07 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236469+0000", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7129999999999999e-06 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236480+0000", 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1352000000000001e-05 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236492+0000", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1411e-05 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.237261+0000", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076960699999999995 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.237298+0000", 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6059000000000002e-05 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.241713+0000", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044153819999999998 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.243066+0000", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001352596 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177647 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.7126047049999999, 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094287539999999993, 2024-09-21T22:52:28.032 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177647 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239719+0000", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239726+0000", 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7819999999999996e-06 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239740+0000", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.418e-05 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239749+0000", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.799e-06 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239818+0000", 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8758999999999997e-05 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239911+0000", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3158000000000005e-05 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.240242+0000", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00033125800000000002 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.241890+0000", 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016481079999999999 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249043+0000", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.007152968 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249104+0000", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0256e-05 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249146+0000", 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2746e-05 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177651 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.693956214, 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071939489999999998, 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177651 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.038 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258368+0000", 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.001e-06 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258376+0000", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5499999999999997e-06 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258393+0000", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6813e-05 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258401+0000", 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5709999999999998e-06 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258464+0000", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2995000000000002e-05 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258572+0000", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107885 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258766+0000", 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000194422 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.259739+0000", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000972503 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265430+0000", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005691008 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265503+0000", 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3322e-05 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265560+0000", 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6879000000000002e-05 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177668 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [watch watch cookie 140232628694864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.26022331700000001, 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0078664040000000005, 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177668 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692100+0000", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4600000000000004e-07 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692103+0000", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6339999999999999e-06 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692114+0000", 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0139999999999999e-05 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692119+0000", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1800000000000004e-06 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692180+0000", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0999999999999999e-05 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692275+0000", 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5364000000000004e-05 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692472+0000", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000197023 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.693277+0000", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080479599999999998 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699813+0000", 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065361899999999999 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699860+0000", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7301e-05 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699966+0000", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010513 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177703 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:28.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.204601963, 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061480559999999998, 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177703 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747722+0000", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0750000000000001e-06 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747727+0000", 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4090000000000002e-06 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747742+0000", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4973e-05 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747751+0000", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7239999999999998e-06 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747788+0000", 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7267000000000001e-05 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747834+0000", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6124000000000001e-05 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747938+0000", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103624 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.748474+0000", 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053669600000000003 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753809+0000", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053350849999999998 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753842+0000", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2271999999999999e-05 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753869+0000", 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.054 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.055 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:28.052+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:28.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T22:52:28.156 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:28.150+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:28.230 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:28.231 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:28.231 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:28.231 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T22:52:28.295 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-21T22:52:28.296+0000 7fc4350cf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26517) UID: 0 2024-09-21T22:52:28.357 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-21T22:52:28.350+0000 7fc7f1943640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-21T22:52:28.385 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:28.385 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:28.385 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:28.385 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:28.385 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T22:52:28.471 INFO:teuthology.orchestra.run.smithi094.stdout:All operations completed A-OK! 2024-09-21T22:52:28.472 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-21T22:52:28.472+0000 7f30bef5e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26519) UID: 0 2024-09-21T22:52:28.533 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:28.533 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T22:52:28.541 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T22:52:28.625 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:146890 3.b 3:d21f0359:::rbd_data.111e2790987.00000000000000ab:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 510979~13309] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:28.626 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.284996007999993, 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.010298846, 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 146890 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285295+0000", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285298+0000", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285319+0000", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0837999999999999e-05 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285345+0000", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6169999999999998e-05 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285359+0000", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4455e-05 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285459+0000", 2024-09-21T22:52:28.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9186999999999997e-05 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.285466+0000", 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1269999999999997e-06 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293602+0000", 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081366660000000007 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:28.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:18.293610+0000", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.293903+0000", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0002927150000001 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.294128+0000", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022479600000000001 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295462+0000", 2024-09-21T22:52:28.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013340439999999999 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295521+0000", 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9747999999999998e-05 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:19.295594+0000", 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2389000000000001e-05 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150735 3.1 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23133641~60884] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.448736394999997, 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.095002048000000006, 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150735 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121555+0000", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.121557+0000", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2230000000000001e-06 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141532+0000", 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019975478000000001 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141574+0000", 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1295999999999999e-05 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141591+0000", 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7e-05 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141696+0000", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105726 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.141921+0000", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022467799999999999 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.143909+0000", 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019877219999999999 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.187369+0000", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043460379 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216447+0000", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.029077457000000001 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216500+0000", 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3220999999999997e-05 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.216557+0000", 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6867999999999998e-05 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150983 3.b 3:d14d2099:::rbd_data.111e4671229f.0000000000000002:1e [read 7471104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.335059380999994, 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078752174999999994, 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150983 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235232+0000", 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235233+0000", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.167e-06 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235238+0000", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0900000000000004e-06 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235282+0000", 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4388999999999998e-05 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235297+0000", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4392999999999999e-05 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313702+0000", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078405328999999996 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313829+0000", 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012659499999999999 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.313984+0000", 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155212 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150984 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334910819000001, 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078686588000000002, 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150984 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235380+0000", 2024-09-21T22:52:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235382+0000", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.694e-06 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235408+0000", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6060000000000001e-05 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235428+0000", 2024-09-21T22:52:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9706000000000001e-05 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235437+0000", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3000000000000007e-06 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314030+0000", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078592829000000003 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314067+0000", 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6999000000000003e-05 2024-09-21T22:52:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150985 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334840439000004, 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078660246000000003, 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150985 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235451+0000", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3300000000000001e-07 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235455+0000", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3270000000000001e-06 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235464+0000", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5629999999999998e-06 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235471+0000", 2024-09-21T22:52:28.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9929999999999998e-06 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314100+0000", 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078629181000000006 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314111+0000", 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0349e-05 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150986 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334810226000002, 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078670128000000006, 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150986 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235481+0000", 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4300000000000003e-07 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235484+0000", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0130000000000002e-06 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235496+0000", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1267000000000001e-05 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235506+0000", 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9469999999999998e-06 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314144+0000", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078637972 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314151+0000", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2860000000000002e-06 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150987 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334773966, 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078680552000000001, 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150987 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235517+0000", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235518+0000", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1600000000000001e-07 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235520+0000", 2024-09-21T22:52:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7470000000000001e-06 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235532+0000", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2147000000000001e-05 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235540+0000", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1430000000000001e-06 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314188+0000", 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078648102999999997 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314198+0000", 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7960000000000007e-06 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150988 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334742273000003, 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078685560000000002, 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150988 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235549+0000", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.37e-07 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235552+0000", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.993e-06 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235562+0000", 2024-09-21T22:52:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2459999999999992e-06 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235572+0000", 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0107e-05 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314227+0000", 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078655623999999993 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314234+0000", 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9530000000000002e-06 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150989 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334710040000004, 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699964999999997, 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150989 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235581+0000", 2024-09-21T22:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235582+0000", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2300000000000001e-07 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235584+0000", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235595+0000", 2024-09-21T22:52:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0448999999999999e-05 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235602+0000", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3200000000000002e-06 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314270+0000", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078667520000000005 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314281+0000", 2024-09-21T22:52:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1165999999999999e-05 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150990 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334679811000001, 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078733973999999998, 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150990 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235611+0000", 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235612+0000", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8999999999999996e-07 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235615+0000", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235627+0000", 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.22e-05 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235639+0000", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1956999999999999e-05 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314338+0000", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078699217000000002 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314345+0000", 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0230000000000004e-06 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:150991 3.b 3.9904b28b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.334643217000007, 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078739977000000003, 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150991 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.669 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235648+0000", 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235652+0000", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235662+0000", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0417e-05 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.235682+0000", 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9380000000000001e-05 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314379+0000", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078697136000000001 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.314388+0000", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0280000000000006e-06 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164804 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:28.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.627266763000002, 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078917910999999993, 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164804 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943024+0000", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943025+0000", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2139999999999999e-06 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943076+0000", 2024-09-21T22:52:28.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0713e-05 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943099+0000", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.283e-05 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943115+0000", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6143000000000001e-05 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021209+0000", 2024-09-21T22:52:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078093625999999999 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021457+0000", 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024862499999999999 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.021942+0000", 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048475999999999999 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164805 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.627156323000001, 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079330240999999996, 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164805 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:28.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943135+0000", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943136+0000", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.02e-06 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943142+0000", 2024-09-21T22:52:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5930000000000002e-06 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943162+0000", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9349999999999999e-05 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943181+0000", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9726000000000001e-05 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022028+0000", 2024-09-21T22:52:28.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.078846374999999996 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022156+0000", 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012826600000000001 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022465+0000", 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030891099999999998 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.680 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164806 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.627079467000002, 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080081608999999998, 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164806 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:28.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943211+0000", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943213+0000", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.593e-06 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943219+0000", 2024-09-21T22:52:28.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9839999999999998e-06 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943243+0000", 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3713000000000001e-05 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943261+0000", 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8056000000000001e-05 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022565+0000", 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.079303881000000007 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.022801+0000", 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000236188 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023293+0000", 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049219400000000005 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164807 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.627011635000002, 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080604386, 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164807 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943279+0000", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943280+0000", 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943287+0000", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3470000000000004e-06 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943309+0000", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.268e-05 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943324+0000", 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4786999999999999e-05 2024-09-21T22:52:28.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023378+0000", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080053448999999999 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023506+0000", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012849500000000001 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023884+0000", 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037764800000000001 2024-09-21T22:52:28.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164808 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.626941365, 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081348826999999999, 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164808 2024-09-21T22:52:28.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943350+0000", 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943351+0000", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9399999999999993e-07 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943357+0000", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2129999999999996e-06 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943379+0000", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2402999999999999e-05 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943411+0000", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1486000000000003e-05 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.023981+0000", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.080570606000000003 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024224+0000", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024312100000000001 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024698+0000", 2024-09-21T22:52:28.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047400399999999998 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164809 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.626856711999999, 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082116660999999994, 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164809 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943434+0000", 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943435+0000", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8299999999999995e-07 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943441+0000", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0299999999999999e-06 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943464+0000", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2503000000000001e-05 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943476+0000", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-05 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.024794+0000", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.081317056999999998 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025037+0000", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243568 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025551+0000", 2024-09-21T22:52:28.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051380999999999996 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164810 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.626796335999998, 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082888419000000005, 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164810 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943495+0000", 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943496+0000", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943501+0000", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7440000000000004e-06 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943520+0000", 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8695999999999999e-05 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943534+0000", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.346e-05 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025646+0000", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082112391000000007 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.025887+0000", 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024081800000000001 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026383+0000", 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00049627999999999999 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164811 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.626737792999997, 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083685124, 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.701 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164811 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943553+0000", 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943554+0000", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1200000000000001e-06 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943571+0000", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6393e-05 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943597+0000", 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943618+0000", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0965999999999999e-05 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026471+0000", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.082852466 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.026719+0000", 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024805600000000003 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027238+0000", 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051931600000000003 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:164812 3.1b 3:def5d1ff:::rbd_data.111e51cd3306.0000000000000000:21 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.626651850000002, 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.084226433000000003, 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:28.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164812 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943639+0000", 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:28.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943641+0000", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4130000000000001e-06 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943647+0000", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4239999999999998e-06 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943666+0000", 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9485999999999999e-05 2024-09-21T22:52:28.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:28.943681+0000", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4222999999999999e-05 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027349+0000", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.083668794000000005 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027507+0000", 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015797900000000001 2024-09-21T22:52:28.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.027866+0000", 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035811399999999998 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:28.709 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T22:52:28.839 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:28.839 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:28.839 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:28.839 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T22:52:28.976 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:28.976 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:28.976 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:28.976 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:28.976 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:28.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T22:52:29.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37886 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.617850144, 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048049794, 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37886 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488379+0000", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488380+0000", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7400000000000003e-07 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488383+0000", 2024-09-21T22:52:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1860000000000001e-06 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488391+0000", 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.488397+0000", 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7640000000000002e-06 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516115+0000", 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.027717795 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.516257+0000", 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014198300000000001 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.536429+0000", 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.020172644 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37967 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61633588399999, 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042070913000000001, 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37967 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489894+0000", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489895+0000", 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.13e-06 2024-09-21T22:52:29.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489900+0000", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.49e-06 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489920+0000", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9964999999999999e-05 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489932+0000", 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1878e-05 2024-09-21T22:52:29.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531639+0000", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041707177999999998 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531766+0000", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012657400000000001 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.531965+0000", 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000198698 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.115 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37968 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 131072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.616282582, 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042247237999999999, 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37968 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.116 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489947+0000", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489948+0000", 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:29.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489952+0000", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.335e-06 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489967+0000", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5058000000000001e-05 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.489986+0000", 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8702000000000002e-05 2024-09-21T22:52:29.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532000+0000", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042014065000000003 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532089+0000", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9192999999999998e-05 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532194+0000", 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104935 2024-09-21T22:52:29.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37970 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 145408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.616227534, 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042486204999999999, 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37970 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490002+0000", 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490003+0000", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490008+0000", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490041+0000", 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3290000000000001e-05 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490054+0000", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2714000000000001e-05 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532227+0000", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042173416999999998 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532320+0000", 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2186000000000001e-05 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532488+0000", 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000168696 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37972 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 155648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61615835000001, 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042690428000000002, 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37972 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490071+0000", 2024-09-21T22:52:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490072+0000", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.032e-06 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490078+0000", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3430000000000003e-06 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490097+0000", 2024-09-21T22:52:29.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9177e-05 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490116+0000", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.893e-05 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532524+0000", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042408082 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532625+0000", 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-21T22:52:29.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532762+0000", 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013627600000000001 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37974 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 165888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61609578300001, 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042881013000000003, 2024-09-21T22:52:29.128 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37974 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490134+0000", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490135+0000", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5199999999999995e-07 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490140+0000", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.276e-06 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490157+0000", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6942000000000001e-05 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490168+0000", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.149e-05 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532796+0000", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042627772000000001 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.532895+0000", 2024-09-21T22:52:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9065000000000001e-05 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533015+0000", 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119516 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37977 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.616046389, 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043126341999999998, 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37977 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490183+0000", 2024-09-21T22:52:29.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490184+0000", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6800000000000009e-07 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490189+0000", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5619999999999997e-06 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490203+0000", 2024-09-21T22:52:29.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4600000000000001e-05 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490214+0000", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1116e-05 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533051+0000", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04283621 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533150+0000", 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-21T22:52:29.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533310+0000", 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015952499999999999 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37979 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61554260600001, 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042872627000000003, 2024-09-21T22:52:29.136 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37979 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490687+0000", 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490689+0000", 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.548e-06 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490696+0000", 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8099999999999998e-06 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490716+0000", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981999999999999e-05 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490732+0000", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5438e-05 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533345+0000", 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042613211999999998 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533412+0000", 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6789e-05 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533560+0000", 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014784800000000001 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37981 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.140 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.615475027, 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043055402, 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37981 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:29.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490755+0000", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490756+0000", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9199999999999999e-07 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490760+0000", 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8799999999999999e-06 2024-09-21T22:52:29.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490775+0000", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5036e-05 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.490786+0000", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0699999999999999e-05 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533594+0000", 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042807647999999997 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533695+0000", 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101099 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533810+0000", 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115047 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37983 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61492578399999, 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042769370000000001, 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37983 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491304+0000", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491306+0000", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491314+0000", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8829999999999995e-06 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491340+0000", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6027999999999999e-05 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491355+0000", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533843+0000", 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042488640000000001 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.533943+0000", 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010001300000000001 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534073+0000", 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129698 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37985 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 245760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.614848303, 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042983237000000001, 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37985 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491381+0000", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491382+0000", 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491388+0000", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4020000000000001e-06 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491402+0000", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4384e-05 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491415+0000", 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.325e-05 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534107+0000", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042692081999999999 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534207+0000", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9253999999999995e-05 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534365+0000", 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157849 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37986 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.614684172, 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043093602000000002, 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37986 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491545+0000", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491547+0000", 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3480000000000001e-06 2024-09-21T22:52:29.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491554+0000", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.182e-06 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491572+0000", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8218000000000001e-05 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491586+0000", 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3842000000000001e-05 2024-09-21T22:52:29.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534401+0000", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042815054999999998 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534502+0000", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100581 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534639+0000", 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013737600000000001 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.156 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37989 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 270336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.614623193, 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043261979999999998, 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37989 2024-09-21T22:52:29.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491606+0000", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491607+0000", 2024-09-21T22:52:29.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4200000000000004e-07 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491612+0000", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6179999999999997e-06 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491625+0000", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3518e-05 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491636+0000", 2024-09-21T22:52:29.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0276e-05 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534673+0000", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043037690000000003 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534771+0000", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7437999999999998e-05 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534868+0000", 2024-09-21T22:52:29.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7497999999999994e-05 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37991 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 280576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.614579581, 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043531031999999997, 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37991 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491650+0000", 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491651+0000", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8800000000000001e-07 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491655+0000", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491670+0000", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5347999999999999e-05 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491681+0000", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0492000000000001e-05 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534895+0000", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043214130000000003 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.534974+0000", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8956999999999996e-05 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535181+0000", 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000206987 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37992 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61453509899999, 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043756998999999998, 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37992 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491694+0000", 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491695+0000", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.16e-07 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491700+0000", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2200000000000003e-06 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491713+0000", 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3412e-05 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491723+0000", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0081999999999999e-05 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535232+0000", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043509142000000001 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535334+0000", 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101705 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535451+0000", 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117522 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37994 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.61448993099999, 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043948055, 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37994 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491740+0000", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.16e-07 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491745+0000", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4329999999999997e-06 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491758+0000", 2024-09-21T22:52:29.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.322e-05 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491768+0000", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9720000000000004e-06 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535505+0000", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043737353 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535585+0000", 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9951999999999996e-05 2024-09-21T22:52:29.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535688+0000", 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102309 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:37996 3.19 3:983aa609:::rbd_data.111ebb10c685.0000000000000007:e [read 339968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.614447634, 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044147860999999997, 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 37996 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491782+0000", 2024-09-21T22:52:29.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491783+0000", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1200000000000002e-07 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491787+0000", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0199999999999996e-06 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491799+0000", 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2396e-05 2024-09-21T22:52:29.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.491812+0000", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.289e-05 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535722+0000", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043909871000000003 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535819+0000", 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:46.535930+0000", 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111161 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:49505 3.0 3:066b28d2:::rbd_data.111e5668b648.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 99.129015045000003, 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.072148748999999998, 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 49505 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977215+0000", 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977217+0000", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6429999999999999e-06 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977230+0000", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2469999999999999e-05 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977261+0000", 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-05 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977276+0000", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4905999999999999e-05 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977450+0000", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017432600000000001 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.977698+0000", 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024771800000000002 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.990297+0000", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012599117999999999 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049165+0000", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058868021999999999 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049199+0000", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4193000000000003e-05 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049288+0000", 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8209999999999997e-05 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.049363+0000", 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5742999999999996e-05 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:151331 3.19 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1070087~53300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.777520570999997, 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049183155999999999, 2024-09-21T22:52:29.182 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151331 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328709+0000", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.328711+0000", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4449999999999999e-06 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341045+0000", 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012333732 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341078+0000", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2638999999999997e-05 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341094+0000", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6566999999999999e-05 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341227+0000", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013307799999999999 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.341361+0000", 2024-09-21T22:52:29.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013343200000000001 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377783+0000", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.036421764000000002 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377833+0000", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0185999999999997e-05 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:23.377892+0000", 2024-09-21T22:52:29.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9312999999999999e-05 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177656 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.276616+0000", 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.8296133779999999, 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4062000989999999, 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:29.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177656 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276616+0000", 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276616+0000", 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276618+0000", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.336e-06 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276625+0000", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9379999999999992e-06 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276640+0000", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4117999999999999e-05 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276649+0000", 2024-09-21T22:52:29.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5049999999999996e-06 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276724+0000", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4639000000000001e-05 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.276741+0000", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7022999999999999e-05 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:26.054622+0000", 2024-09-21T22:52:29.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.77788142900000001 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:26.054665+0000", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2339999999999998e-05 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.682390+0000", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6277258240000001 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.682597+0000", 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020670500000000001 2024-09-21T22:52:29.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.682816+0000", 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000219242 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.192 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:29.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T22:52:29.359 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:29.360 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:29.360 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:29.360 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:29.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T22:52:29.500 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:29.500 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:29.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49451 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'414, mlcod=53'412)", 2024-09-21T22:52:29.637 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 99.804570362999996, 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.019228213000000001, 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831864+0000", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.831865+0000", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.59e-06 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834124+0000", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002258323 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834153+0000", 2024-09-21T22:52:29.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9612999999999999e-05 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834166+0000", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2901000000000001e-05 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834252+0000", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5317999999999999e-05 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.834279+0000", 2024-09-21T22:52:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7385e-05 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851049+0000", 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016770397999999999 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:49.851092+0000", 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2685000000000002e-05 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.641 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49511 3.4 e55/24 3:22a0c71f:::rbd_data.111e5668b648.000000000000000b:head v 55'331, mlcod=55'329)", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 99.547145493000002, 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046013640000000001, 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089289+0000", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089292+0000", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8200000000000001e-06 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089312+0000", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.067e-05 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089357+0000", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5111999999999998e-05 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089374+0000", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6327000000000001e-05 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089588+0000", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021470199999999999 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.089644+0000", 2024-09-21T22:52:29.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5578999999999998e-05 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093756+0000", 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041118190000000001 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.093890+0000", 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013433499999999999 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.645 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:49678 3.0 e55/24 3:043e359c:::rbd_data.111e5668b648.0000000000000001:head v 55'423, mlcod=55'421)", 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 99.216807424999999, 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048542780000000001, 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419627+0000", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419629+0000", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8640000000000001e-06 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419645+0000", 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6364000000000001e-05 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419672+0000", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6849999999999999e-05 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419685+0000", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2683999999999999e-05 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419762+0000", 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7318999999999996e-05 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.419786+0000", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4569000000000001e-05 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424408+0000", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00462116 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:50.424481+0000", 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3467999999999995e-05 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:53976 3.4 e59/24 3:27b9fd12:::rbd_data.111e5a344aa.0000000000000114:head v 59'349, mlcod=56'346)", 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 94.021106109000002, 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004563213, 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615328+0000", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615330+0000", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.305e-06 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615346+0000", 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5892e-05 2024-09-21T22:52:29.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615388+0000", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1984999999999999e-05 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615408+0000", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.942e-05 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615539+0000", 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013105899999999999 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.615589+0000", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9885000000000001e-05 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619786+0000", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041974409999999997 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.619891+0000", 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105226 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:54009 3.4 e59/24 3:24391d1a:::rbd_data.111e5a344aa.000000000000001c:head v 59'353, mlcod=59'349)", 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.989986254000002, 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050932349999999998, 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:52:29.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646448+0000", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646450+0000", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-06 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646462+0000", 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1977000000000001e-05 2024-09-21T22:52:29.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646488+0000", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5774000000000001e-05 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646502+0000", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4494999999999999e-05 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646646+0000", 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014328 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.646708+0000", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000004e-05 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651437+0000", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047288989999999999 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:50:55.651541+0000", 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010401 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:165039 3.0 e84/24 3:03211d2a:::rbd_data.111ede3bcd88.000000000000003a:head v 84'564, mlcod=82'562)", 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.539912469999997, 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047176889999999997, 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:52:29.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096522+0000", 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.096523+0000", 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.683e-06 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097147+0000", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062348899999999997 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097177+0000", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0453000000000001e-05 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097207+0000", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.976e-05 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097390+0000", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018328800000000001 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.097463+0000", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2529999999999998e-05 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101141+0000", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036776109999999999 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:29.101239+0000", 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8875000000000005e-05 2024-09-21T22:52:29.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.4.0:2479 3.9 e105/24 3:91cf30d1:::rbd_data.111e6870f364.0000000000000066:1b v 105'772, mlcod=100'768)", 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.041773+0000", 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.594661217000002, 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0049493180000000003, 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041773+0000", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041773+0000", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041774+0000", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5600000000000005e-07 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041782+0000", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3899999999999993e-06 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041807+0000", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5360000000000001e-05 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.041824+0000", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.042020+0000", 2024-09-21T22:52:29.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000195572 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.042074+0000", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4540000000000003e-05 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.046602+0000", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045273980000000002 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.046722+0000", 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012076199999999999 2024-09-21T22:52:29.665 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1751 3.4 e105/24 3:24acba3c:::rbd_data.111e6870f364.00000000000001c6:1b v 105'547, mlcod=103'543)", 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.049999+0000", 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.586435389999998, 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046994000000000003, 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049999+0000", 2024-09-21T22:52:29.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.049999+0000", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050001+0000", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.367e-06 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050015+0000", 2024-09-21T22:52:29.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4153e-05 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050043+0000", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.724e-05 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050060+0000", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7249999999999999e-05 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050307+0000", 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024684799999999998 2024-09-21T22:52:29.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050360+0000", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3276000000000003e-05 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054576+0000", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042159370000000003 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054698+0000", 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012232899999999999 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1775 3.0 e110/24 3:0443c2ac:::rbd_data.111ec49ace6d.0000000000000008:16 v 110'601, mlcod=105'597)", 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.063557+0000", 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.572876717, 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054499379999999997, 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063557+0000", 2024-09-21T22:52:29.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063557+0000", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063558+0000", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0999999999999997e-07 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063599+0000", 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0463000000000002e-05 2024-09-21T22:52:29.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063619+0000", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0426999999999999e-05 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063637+0000", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.736e-05 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063841+0000", 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020469100000000001 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.063880+0000", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.841e-05 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.068895+0000", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005015294 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.069007+0000", 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112483 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1826 3.0 e118/24 3:04a21c61:::rbd_data.111e4aac08c7.0000000000000335:10 v 118'614, mlcod=116'611)", 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.110744+0000", 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.52569059, 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045818339999999999, 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110744+0000", 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110744+0000", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110744+0000", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.75e-07 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110752+0000", 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8299999999999996e-06 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110829+0000", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7334000000000002e-05 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110841+0000", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.181e-05 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110942+0000", 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010104599999999999 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.110967+0000", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4477999999999998e-05 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.115278+0000", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043115970000000003 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.115325+0000", 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7264e-05 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.677 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1827 3.4 e118/24 3:20487b15:::rbd_data.111e4aac08c7.00000000000002b6:10 v 118'575, mlcod=115'572)", 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.111698+0000", 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 20.524736288, 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0046522710000000004, 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111698+0000", 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111698+0000", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111699+0000", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.61e-07 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111705+0000", 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6930000000000001e-06 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111716+0000", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0848e-05 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111722+0000", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6280000000000002e-06 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111814+0000", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2236000000000003e-05 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.111840+0000", 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5888999999999999e-05 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.116311+0000", 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044711999999999998 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.116350+0000", 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9016e-05 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.4.0:2600 3.9 e119/24 3:903684e3:::rbd_data.111e38708da6.0000000000000085:f v 119'791, mlcod=118'787)", 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.117663+0000", 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.518771499, 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0068981900000000002, 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117663+0000", 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117663+0000", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117664+0000", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117677+0000", 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3006999999999999e-05 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117731+0000", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4070000000000002e-05 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117757+0000", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5856000000000001e-05 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117918+0000", 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016120899999999999 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.117977+0000", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9206000000000002e-05 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.124446+0000", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064687629999999998 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.124561+0000", 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115159 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.4.0:2602 3.9 e119/24 3:92c9c0a2:::rbd_data.111e38708da6.000000000000020c:f v 119'793, mlcod=118'787)", 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.125717+0000", 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.51071718, 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070989360000000001, 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125717+0000", 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125717+0000", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125719+0000", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4269999999999998e-06 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125737+0000", 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7286e-05 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125768+0000", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1013000000000003e-05 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125787+0000", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8986000000000001e-05 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126069+0000", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028187500000000001 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126123+0000", 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3919999999999999e-05 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.132443+0000", 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063208250000000004 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.132816+0000", 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037260400000000001 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1844 3.4 e119/24 3:226e6cf8:::rbd_data.111e38708da6.0000000000000029:f v 119'588, mlcod=118'585)", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.125859+0000", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.510575631000002, 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0081136309999999996, 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125859+0000", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125859+0000", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125859+0000", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0999999999999998e-07 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125873+0000", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3903000000000001e-05 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125894+0000", 2024-09-21T22:52:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1172999999999999e-05 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125909+0000", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4263e-05 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126301+0000", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039215799999999998 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126353+0000", 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2281999999999999e-05 2024-09-21T22:52:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.133872+0000", 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0075186080000000004 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.133972+0000", 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100634 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1845 3.0 e119/24 3:006bafd0:::rbd_data.111e38708da6.00000000000000bc:f v 119'629, mlcod=118'626)", 2024-09-21T22:52:29.693 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.125954+0000", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.510480542, 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082489739999999992, 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125954+0000", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125954+0000", 2024-09-21T22:52:29.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125954+0000", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9699999999999995e-07 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125974+0000", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9363000000000001e-05 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.125994+0000", 2024-09-21T22:52:29.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0183e-05 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126012+0000", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7859999999999998e-05 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126130+0000", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117983 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126185+0000", 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5216000000000001e-05 2024-09-21T22:52:29.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.134094+0000", 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079089030000000001 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.134203+0000", 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108669 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1846 3.4 e119/24 3:2658c528:::rbd_data.111e38708da6.0000000000000099:f v 119'590, mlcod=118'585)", 2024-09-21T22:52:29.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.126050+0000", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.510384692999999, 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094622899999999999, 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126050+0000", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126050+0000", 2024-09-21T22:52:29.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126050+0000", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1399999999999995e-07 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126059+0000", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8489999999999995e-06 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126077+0000", 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8009999999999999e-05 2024-09-21T22:52:29.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.126090+0000", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3169999999999999e-05 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.130813+0000", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047225399999999999 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.130869+0000", 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5612999999999998e-05 2024-09-21T22:52:29.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.135414+0000", 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004545777 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.135512+0000", 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7417000000000006e-05 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.701 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1847 3.0 e119/24 3:06c17af1:::rbd_data.111e38708da6.000000000000015b:f v 119'631, mlcod=118'626)", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.131174+0000", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.505260272000001, 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054054339999999998, 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131174+0000", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131174+0000", 2024-09-21T22:52:29.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131177+0000", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.756e-06 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131221+0000", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3930000000000001e-05 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131928+0000", 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070764299999999999 2024-09-21T22:52:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.131952+0000", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3728000000000001e-05 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.132394+0000", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00044226399999999999 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.132466+0000", 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1816000000000005e-05 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.136478+0000", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040123549999999996 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.136579+0000", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100942 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1850 3.0 e119/24 3:074b6750:::rbd_data.111e38708da6.0000000000000157:f v 119'635, mlcod=119'629)", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.145634+0000", 2024-09-21T22:52:29.705 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.490799914, 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072717459999999999, 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145634+0000", 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145634+0000", 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145636+0000", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8700000000000001e-06 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145651+0000", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4403000000000001e-05 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145688+0000", 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7549999999999998e-05 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145709+0000", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1007000000000001e-05 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145920+0000", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021125199999999999 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.145980+0000", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9783e-05 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.152794+0000", 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0068139020000000002 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.152906+0000", 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011197900000000001 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.4.0:2608 3.9 e119/24 3:976702ba:::rbd_data.111e38708da6.000000000000019d:f v 119'799, mlcod=119'795)", 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.148502+0000", 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.487931890999999, 2024-09-21T22:52:29.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047297950000000002, 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148502+0000", 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148502+0000", 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148504+0000", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.99e-06 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148519+0000", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4837e-05 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148554+0000", 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4983e-05 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148574+0000", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9769999999999999e-05 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148705+0000", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000131061 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.148758+0000", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2673999999999997e-05 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.153122+0000", 2024-09-21T22:52:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004364461 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.153232+0000", 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110019 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1867 3.0 e120/24 3:07720949:::rbd_data.111ebb10c685.000000000000002d:e v 120'639, mlcod=119'636)", 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:11.047241+0000", 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.589192916999998, 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048136619999999998, 2024-09-21T22:52:29.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047241+0000", 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047241+0000", 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047242+0000", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047260+0000", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7652999999999999e-05 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047280+0000", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9942999999999999e-05 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.047294+0000", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4333e-05 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.048531+0000", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001236425 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.048622+0000", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1316000000000005e-05 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.051899+0000", 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003277439 2024-09-21T22:52:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:11.052055+0000", 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015563600000000001 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:29.717 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:29.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T22:52:29.848 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:29.848 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:29.848 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:29.848 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:29.862 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T22:52:29.979 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:29.979 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:29.979 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:29.979 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:29.979 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:29.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21057 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:30.118 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135850891, 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055705257000000001, 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21057 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979289+0000", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979290+0000", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8999999999999995e-07 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979293+0000", 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979303+0000", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979310+0000", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2400000000000001e-06 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034893+0000", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055582845999999998 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034937+0000", 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4558999999999998e-05 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.034994+0000", 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7059999999999999e-05 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21058 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.13582089000001, 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055937244999999997, 2024-09-21T22:52:30.122 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21058 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979319+0000", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979320+0000", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2399999999999998e-07 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979323+0000", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0259999999999999e-06 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979331+0000", 2024-09-21T22:52:30.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6160000000000002e-06 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979338+0000", 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035048+0000", 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055709802000000003 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035152+0000", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104331 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035256+0000", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104139 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21059 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:30.126 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.13579318399999, 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056044178, 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21059 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979347+0000", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979348+0000", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979353+0000", 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979374+0000", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0778999999999999e-05 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979382+0000", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035292+0000", 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055909219000000003 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035337+0000", 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5522999999999998e-05 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035391+0000", 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3918000000000001e-05 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21060 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.130 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.13574723399999, 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056261101000000001, 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21060 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:30.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979393+0000", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979394+0000", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7400000000000002e-07 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979406+0000", 2024-09-21T22:52:30.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2303000000000001e-05 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979416+0000", 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979423+0000", 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1439999999999997e-06 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035441+0000", 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056017662000000003 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035550+0000", 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010938300000000001 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035654+0000", 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21061 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 268288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135707099, 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056356187000000002, 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21061 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979433+0000", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979434+0000", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6300000000000005e-07 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979437+0000", 2024-09-21T22:52:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979447+0000", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0553e-05 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979454+0000", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9029999999999998e-06 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035691+0000", 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056237114999999997 2024-09-21T22:52:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035735+0000", 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3714999999999997e-05 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035789+0000", 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4261000000000001e-05 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21062 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 278528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.1356771, 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056593218000000001, 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21062 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979463+0000", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979464+0000", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.13e-07 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979466+0000", 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8830000000000002e-06 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979481+0000", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4496e-05 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979488+0000", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0160000000000003e-06 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035842+0000", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056354453999999998 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.035949+0000", 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010627499999999999 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036056+0000", 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107481 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21063 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135643039, 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056692613000000003, 2024-09-21T22:52:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21063 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979497+0000", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979498+0000", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979501+0000", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0730000000000001e-06 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979522+0000", 2024-09-21T22:52:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1579000000000001e-05 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979530+0000", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9459999999999998e-06 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036092+0000", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056561293999999998 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036136+0000", 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4234999999999999e-05 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036190+0000", 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3816000000000001e-05 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21064 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135599184, 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056903294, 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21064 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979541+0000", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979542+0000", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6000000000000003e-07 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979545+0000", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.579e-06 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979559+0000", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4143e-05 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979567+0000", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.109e-06 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036235+0000", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056668582000000002 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036340+0000", 2024-09-21T22:52:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104854 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036444+0000", 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104267 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21065 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 313344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135563053, 2024-09-21T22:52:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057094499, 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21065 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979577+0000", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979578+0000", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979581+0000", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.92e-06 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979606+0000", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5125000000000001e-05 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979615+0000", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036486+0000", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056871009 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036571+0000", 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4643999999999998e-05 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036672+0000", 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010089099999999999 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21066 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 337920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135512472, 2024-09-21T22:52:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057315873000000003, 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21066 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979628+0000", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9299999999999997e-07 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979632+0000", 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8430000000000003e-06 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979645+0000", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2713e-05 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979664+0000", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439000000000001e-05 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036739+0000", 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057074749000000001 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036836+0000", 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7195999999999996e-05 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036944+0000", 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010713999999999999 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21067 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135407258, 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057442528999999999, 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21067 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979733+0000", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979734+0000", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2699999999999999e-06 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979740+0000", 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9560000000000002e-06 2024-09-21T22:52:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979760+0000", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9712999999999999e-05 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979778+0000", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7898999999999999e-05 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.036989+0000", 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057210971999999999 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037079+0000", 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889000000000006e-05 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037175+0000", 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6830000000000006e-05 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21068 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135343748, 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057633644999999997, 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21068 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979796+0000", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979797+0000", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.06e-06 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979803+0000", 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-21T22:52:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979837+0000", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4801000000000001e-05 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979875+0000", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280999999999998e-05 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037239+0000", 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057364765999999998 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037325+0000", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5611000000000004e-05 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037430+0000", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010490899999999999 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21069 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.13524554599999, 2024-09-21T22:52:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057770324999999997, 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21069 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979895+0000", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979896+0000", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979902+0000", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979926+0000", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4321999999999999e-05 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979941+0000", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5058999999999999e-05 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037472+0000", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057530132999999997 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037559+0000", 2024-09-21T22:52:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7769000000000006e-05 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037665+0000", 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105572 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21070 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135180283, 2024-09-21T22:52:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057967566999999998, 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21070 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979960+0000", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979961+0000", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979967+0000", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8529999999999997e-06 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.979990+0000", 2024-09-21T22:52:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3592e-05 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980004+0000", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4063000000000001e-05 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037716+0000", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057711771000000002 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037821+0000", 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104566 2024-09-21T22:52:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037927+0000", 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106612 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21071 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135114366, 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058082027000000001, 2024-09-21T22:52:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21071 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980026+0000", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980027+0000", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0759999999999999e-06 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980033+0000", 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8969999999999996e-06 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980067+0000", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4693999999999997e-05 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980081+0000", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3713000000000001e-05 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.037972+0000", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057891317999999997 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038034+0000", 2024-09-21T22:52:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1179999999999994e-05 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038108+0000", 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4148999999999998e-05 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21072 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.135037264, 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058249759999999998, 2024-09-21T22:52:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21072 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980103+0000", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980104+0000", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.234e-06 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980121+0000", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6731999999999998e-05 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980140+0000", 2024-09-21T22:52:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8899e-05 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980158+0000", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7873e-05 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038155+0000", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.057997719000000003 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038246+0000", 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0580000000000004e-05 2024-09-21T22:52:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038353+0000", 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106723 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:21073 3.1a 3:5e8ca24d:::rbd_data.111ebb10c685.0000000000000051:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.134963664, 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058313306000000002, 2024-09-21T22:52:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21073 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980176+0000", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980178+0000", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1069999999999999e-06 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980183+0000", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.412e-06 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980200+0000", 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309999999999999e-05 2024-09-21T22:52:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:41.980213+0000", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2719e-05 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038382+0000", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.058169303999999998 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038430+0000", 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620000000000001e-05 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:42.038490+0000", 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9833999999999997e-05 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150127 3.11 e81/24 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head v 81'816, mlcod=79'813)", 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.249862739999998, 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.085138597999999996, 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865277+0000", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.865280+0000", 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9859999999999999e-06 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884772+0000", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019491715999999999 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884815+0000", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309000000000002e-05 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.884835+0000", 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9766999999999999e-05 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885029+0000", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019343600000000001 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.885068+0000", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9688999999999999e-05 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950352+0000", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065283806 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950416+0000", 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3888999999999998e-05 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4382.0:150735 3.1 e81/24 3:84799f6d:::rbd_data.111edf735d75.0000000000000001:head v 81'307, mlcod=80'305)", 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.971002163999998, 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.071776031000000004, 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144138+0000", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.144140+0000", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170033+0000", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025892330000000002 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170070+0000", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6828e-05 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170091+0000", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000001e-05 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170209+0000", 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011801199999999999 2024-09-21T22:52:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.170261+0000", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159e-05 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215778+0000", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045516503 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:23.215914+0000", 2024-09-21T22:52:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136205 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:167676 3.9 3:90aeb8c8:::rbd_data.111ee98c1da3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.885243164999999, 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.074107391999999994, 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167676 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229897+0000", 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229899+0000", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.086e-06 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229910+0000", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1046999999999999e-05 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229941+0000", 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0868999999999999e-05 2024-09-21T22:52:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.229956+0000", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4893e-05 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230103+0000", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000147439 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.230341+0000", 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023779300000000001 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.244398+0000", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014056587000000001 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303780+0000", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059382193999999999 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303813+0000", 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3339e-05 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.303911+0000", 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8004999999999994e-05 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:34.304004+0000", 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3140000000000006e-05 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.199 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:30.200 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T22:52:30.273 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T22:52:30.273 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-21T22:52:30.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-21T22:52:30.277 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-21T22:52:30.310 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T22:52:30.334 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:30.334 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:30.334 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:30.334 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:30.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T22:52:30.465 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:30.465 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:30.465 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:30.465 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:30.465 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:30.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T22:52:30.591 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-21T22:52:30.588+0000 7fe4f0d43640 -1 mon.a@0(leader).osd e139 definitely_dead 0 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132636 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3852288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323458235000004, 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066449200999999999, 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132636 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276513+0000", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276514+0000", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276518+0000", 2024-09-21T22:52:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7079999999999999e-06 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276529+0000", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1207e-05 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276539+0000", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0101999999999999e-05 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276566+0000", 2024-09-21T22:52:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7372000000000002e-05 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276569+0000", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.279650+0000", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030816440000000001 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342733+0000", 2024-09-21T22:52:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063083252000000006 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.342962+0000", 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888100000000001 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132637 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3854336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323416389000002, 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066978432000000004, 2024-09-21T22:52:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132637 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276555+0000", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276556+0000", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5099999999999999e-07 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276563+0000", 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1040000000000001e-06 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276574+0000", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1790999999999999e-05 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276584+0000", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0799999999999995e-06 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276613+0000", 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9648000000000001e-05 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276616+0000", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4990000000000001e-06 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343085+0000", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066469156000000001 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343322+0000", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023677099999999999 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343533+0000", 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021173200000000001 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132638 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3834178~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323370914999998, 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067475835999999997, 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132638 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276600+0000", 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276601+0000", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018e-06 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276606+0000", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276617+0000", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1447999999999999e-05 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276624+0000", 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8190000000000003e-06 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276649+0000", 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4332e-05 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276651+0000", 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4679999999999999e-06 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343641+0000", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066989516999999998 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.343869+0000", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022888199999999999 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344076+0000", 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020682200000000001 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132639 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3835904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323335986999993, 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067748597999999993, 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132639 2024-09-21T22:52:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276635+0000", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276636+0000", 2024-09-21T22:52:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2699999999999999e-07 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276640+0000", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276652+0000", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1287999999999999e-05 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276657+0000", 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8839999999999999e-06 2024-09-21T22:52:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276681+0000", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3668000000000001e-05 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276684+0000", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.264e-06 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344147+0000", 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067462667000000004 2024-09-21T22:52:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344265+0000", 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001176 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344384+0000", 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011932800000000001 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132640 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3856384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323303741000004, 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068066264000000001, 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132640 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276668+0000", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-07 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276672+0000", 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269999999999996e-06 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276683+0000", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.101e-05 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276692+0000", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7280000000000001e-06 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276716+0000", 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4176999999999999e-05 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276719+0000", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6340000000000001e-06 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344466+0000", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067746878999999996 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344618+0000", 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000152552 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344734+0000", 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115435 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132641 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3837952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323268009000003, 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068325038000000005, 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132641 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276703+0000", 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276704+0000", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7400000000000003e-07 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276708+0000", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276718+0000", 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0044000000000001e-05 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276726+0000", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5580000000000003e-06 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276755+0000", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9169000000000001e-05 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276757+0000", 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-21T22:52:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344803+0000", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068045545999999998 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.344920+0000", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011701699999999999 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345028+0000", 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108451 2024-09-21T22:52:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132643 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3840000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323230832999997, 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068584952000000005, 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132643 2024-09-21T22:52:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276740+0000", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276741+0000", 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8599999999999997e-07 2024-09-21T22:52:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276746+0000", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276758+0000", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2092e-05 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276764+0000", 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-21T22:52:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276787+0000", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2687000000000001e-05 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.508e-06 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345094+0000", 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068304889999999993 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345204+0000", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345325+0000", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121144 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:132650 3.16 3:68034965:::rbd_data.111eaf8fb188.0000000000000003:1a [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.323196940000003, 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821406000000002, 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 132650 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276774+0000", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276775+0000", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276778+0000", 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4350000000000001e-06 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276789+0000", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0403e-05 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276797+0000", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.337e-06 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276821+0000", 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3825e-05 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.276824+0000", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6089999999999999e-06 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345370+0000", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068546739999999995 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345488+0000", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117955 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:15.345596+0000", 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010753 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:146893 3.2 3:45accec5:::rbd_header.111e2790987:head [watch reconnect cookie 140232427467856 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 72.314761595999997, 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0211940610000001, 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146893 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285210+0000", 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285211+0000", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.579e-06 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285218+0000", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1620000000000002e-06 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285236+0000", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7793999999999999e-05 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285245+0000", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9900000000000003e-06 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285312+0000", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6649999999999994e-05 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.285317+0000", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292378+0000", 2024-09-21T22:52:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070603100000000002 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:18.292387+0000", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8439999999999993e-06 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.303991+0000", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011603129 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.304410+0000", 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041929400000000001 2024-09-21T22:52:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:19.306404+0000", 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019939179999999999 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164847 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1671168~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645693244, 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063018437999999996, 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164847 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954278+0000", 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954279+0000", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.004e-06 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954284+0000", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954300+0000", 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6529999999999999e-05 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954313+0000", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2343e-05 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016839+0000", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.062526478999999996 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.016931+0000", 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2095999999999994e-05 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017297+0000", 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036510999999999998 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164848 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1720320~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645641941000001, 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063833261000000002, 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164848 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954329+0000", 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954330+0000", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.02e-06 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954336+0000", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.716e-06 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954354+0000", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8349999999999999e-05 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954369+0000", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4552999999999999e-05 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017376+0000", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063006896000000007 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.017598+0000", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000222461 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018163+0000", 2024-09-21T22:52:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00056426499999999999 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164849 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1769472~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645584024999998, 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064291582, 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164849 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954387+0000", 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954388+0000", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1000000000000001e-06 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954394+0000", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8560000000000003e-06 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954412+0000", 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.768e-05 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954426+0000", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.38e-05 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018205+0000", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.063779695999999997 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018297+0000", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2059999999999996e-05 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018679+0000", 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038139000000000001 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164850 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1818624~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645528366000001, 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065159506000000006, 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164850 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954443+0000", 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954444+0000", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0160000000000001e-06 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954450+0000", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954469+0000", 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-21T22:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954483+0000", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3499999999999999e-05 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.018764+0000", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.064281192000000001 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019006+0000", 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000242308 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019602+0000", 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059597999999999997 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164851 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1867776~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645472216999998, 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065615952000000005, 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164851 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954499+0000", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954500+0000", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954514+0000", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4326e-05 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954532+0000", 2024-09-21T22:52:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.738e-05 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954545+0000", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019670+0000", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065124586999999998 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.019773+0000", 2024-09-21T22:52:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102636 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020115+0000", 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034243300000000001 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164852 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645409725, 2024-09-21T22:52:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066386772999999996, 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164852 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954562+0000", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954563+0000", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-06 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954568+0000", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954584+0000", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6880000000000001e-05 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954598+0000", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3397000000000001e-05 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020195+0000", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065597576000000005 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020414+0000", 2024-09-21T22:52:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021861100000000001 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020948+0000", 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053436300000000005 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164853 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 1966080~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645308907, 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066807912999999997, 2024-09-21T22:52:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164853 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954662+0000", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954663+0000", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.023e-06 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954669+0000", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954687+0000", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8315999999999999e-05 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954701+0000", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.020988+0000", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066286269999999994 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021080+0000", 2024-09-21T22:52:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2223000000000001e-05 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021470+0000", 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039038099999999997 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164854 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2015232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645253087999997, 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067631472999999998, 2024-09-21T22:52:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164854 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954718+0000", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954719+0000", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.24e-06 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954724+0000", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954742+0000", 2024-09-21T22:52:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7099999999999999e-05 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954755+0000", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3626e-05 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021546+0000", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066790568999999994 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.021794+0000", 2024-09-21T22:52:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024777099999999998 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022350+0000", 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055616400000000005 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:164855 3.2 3:47381ee8:::rbd_data.111e51cd3306.0000000000000001:21 [read 2064384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.645200381999999, 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068110181000000006, 2024-09-21T22:52:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 164855 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954771+0000", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954772+0000", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1699999999999997e-07 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954777+0000", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.079e-06 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954793+0000", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6540000000000001e-05 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:28.954807+0000", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3113e-05 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022387+0000", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067580310000000005 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022483+0000", 2024-09-21T22:52:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5990999999999996e-05 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:29.022881+0000", 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039823099999999998 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170366 3.2 3:444a0894:::rbd_data.111e470eb4b3.0000000000000009:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.097106709000002, 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96409512500000005, 2024-09-21T22:52:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170366 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502865+0000", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502868+0000", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.326e-06 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502882+0000", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3703000000000001e-05 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502912+0000", 2024-09-21T22:52:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.502928+0000", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5605999999999999e-05 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503062+0000", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134487 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.503081+0000", 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8462999999999999e-05 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463612+0000", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96053139899999995 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.463937+0000", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.464357+0000", 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042014100000000001 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466319+0000", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001962732 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466653+0000", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033388200000000003 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466755+0000", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101952 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:43.466960+0000", 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000204465 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:177495 3.1d 3:bb619233:::rbd_object_map.111ea6ec99f1:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:52:23.224392+0000", 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.3755789839999997, 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.015946859, 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 177495 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224392+0000", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224392+0000", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224395+0000", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3259999999999998e-06 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224408+0000", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3413e-05 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224443+0000", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5040000000000003e-05 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224473+0000", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0343000000000001e-05 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224603+0000", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129352 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:23.224622+0000", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9432999999999998e-05 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.229745+0000", 2024-09-21T22:52:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0051223659999999 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.230140+0000", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000395716 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.230631+0000", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00049098999999999996 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.239789+0000", 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091574099999999995 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.240033+0000", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024407000000000001 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.240208+0000", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017483200000000001 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:52:24.240339+0000", 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013156800000000001 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:30.694 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:30.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T22:52:30.820 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:30.820 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:30.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T22:52:31.007 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:31.008 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:31.008 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:31.008 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:31.008 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:31.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T22:52:31.148 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.21027631699999, 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.97077930099999998, 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4207", 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3519759960", 2024-09-21T22:52:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935296+0000", 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935298+0000", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935326+0000", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.743e-05 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935351+0000", 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5049999999999999e-05 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935376+0000", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4859999999999999e-05 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935489+0000", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113569 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:52.935505+0000", 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5613000000000001e-05 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.902984+0000", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.96747953900000006 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903423+0000", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043868999999999999 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.903959+0000", 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000535572 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905507+0000", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015481049999999999 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.905918+0000", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041107699999999998 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906014+0000", 2024-09-21T22:52:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6335999999999994e-05 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:49:53.906076+0000", 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37721 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 393216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66520445800001, 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096862227999999995, 2024-09-21T22:52:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37721 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480368+0000", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480369+0000", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.116e-06 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480376+0000", 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480395+0000", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8632000000000001e-05 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480414+0000", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8859e-05 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.576902+0000", 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096487892000000006 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577055+0000", 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153673 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577230+0000", 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017514900000000001 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37723 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66514100800001, 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097469207000000002, 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37723 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480432+0000", 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480433+0000", 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8700000000000004e-07 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480438+0000", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480459+0000", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1036e-05 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480475+0000", 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209e-05 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577311+0000", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096836202999999996 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577516+0000", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020463600000000001 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577901+0000", 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038497299999999999 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37725 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66507783, 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097917523000000006, 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37725 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480495+0000", 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480496+0000", 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9000000000000005e-07 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480501+0000", 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1359999999999996e-06 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480523+0000", 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1942000000000001e-05 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480537+0000", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.577974+0000", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097437365999999997 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578167+0000", 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192888 2024-09-21T22:52:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578412+0000", 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024499400000000001 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37727 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66502149, 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098527711000000004, 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37727 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480551+0000", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480552+0000", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999999999999998e-07 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480556+0000", 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239999999999998e-06 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480568+0000", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2356e-05 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480577+0000", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9889999999999998e-06 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578481+0000", 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.097904440999999995 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.578678+0000", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019666099999999999 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579079+0000", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040083000000000002 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37729 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66497974799999, 2024-09-21T22:52:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098925561999999995, 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37729 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480593+0000", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480594+0000", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5999999999999991e-07 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480599+0000", 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480620+0000", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1009000000000002e-05 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480633+0000", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3293e-05 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579151+0000", 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098518122999999999 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579346+0000", 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019502300000000001 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579518+0000", 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017203800000000001 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37733 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66491914700001, 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099304042999999995, 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37733 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480654+0000", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480655+0000", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1659999999999999e-06 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480660+0000", 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-21T22:52:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480697+0000", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6341999999999999e-05 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480713+0000", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579586+0000", 2024-09-21T22:52:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.098872398 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579779+0000", 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000192945 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.579958+0000", 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017888299999999999 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37735 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.664838711, 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099665526000000004, 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37735 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480734+0000", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480735+0000", 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.133e-06 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480752+0000", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6463000000000001e-05 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480773+0000", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1841999999999999e-05 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480789+0000", 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5879000000000002e-05 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580081+0000", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099291897000000004 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580235+0000", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015413300000000001 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580399+0000", 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164179 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37738 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66476260100001, 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100011904, 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37738 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480810+0000", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480811+0000", 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960000000000001e-06 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480832+0000", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1140000000000001e-05 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480851+0000", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9021999999999998e-05 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480865+0000", 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3569e-05 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580488+0000", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099623166999999999 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580664+0000", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175797 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580822+0000", 2024-09-21T22:52:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015801300000000001 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37753 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.664675528, 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100258243, 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37753 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480897+0000", 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480898+0000", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7000000000000003e-07 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480902+0000", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.856e-06 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480918+0000", 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480928+0000", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0219e-05 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.580902+0000", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.099973838999999995 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581032+0000", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012985399999999999 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581155+0000", 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012382199999999999 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37755 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66463204, 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100659231, 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37755 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480941+0000", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2699999999999998e-07 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480945+0000", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.129e-06 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480954+0000", 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7729999999999994e-06 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480962+0000", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0169999999999996e-06 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581233+0000", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100270279 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581426+0000", 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019289900000000001 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581600+0000", 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017430700000000001 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37756 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.664598402, 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10099258, 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37756 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480974+0000", 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480975+0000", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5000000000000002e-07 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480978+0000", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0170000000000001e-06 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480987+0000", 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.480996+0000", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1230000000000006e-06 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581698+0000", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.100701923 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581839+0000", 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014098999999999999 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.581967+0000", 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012751099999999999 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37758 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 542720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66456463599999, 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101411238, 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37758 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481008+0000", 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481009+0000", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481012+0000", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.027e-06 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481021+0000", 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0690000000000008e-06 2024-09-21T22:52:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481030+0000", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3929999999999997e-06 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582059+0000", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.10102878799999999 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582250+0000", 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019146899999999999 2024-09-21T22:52:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582419+0000", 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168856 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37760 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 567296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66453335200001, 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101696802, 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37760 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481039+0000", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481040+0000", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4000000000000001e-07 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481043+0000", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.004e-06 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481057+0000", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4099e-05 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481074+0000", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7039999999999999e-05 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582492+0000", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101418304 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582622+0000", 2024-09-21T22:52:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012961 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582736+0000", 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011410500000000001 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37762 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 577536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66448799699999, 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102095698, 2024-09-21T22:52:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37762 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481085+0000", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1299999999999999e-07 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481088+0000", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1200000000000002e-06 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481100+0000", 2024-09-21T22:52:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1807000000000001e-05 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481107+0000", 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9090000000000003e-06 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.582818+0000", 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.101710802 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583010+0000", 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019215399999999999 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583180+0000", 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017019299999999999 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37764 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.664456501, 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102375774, 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37764 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481116+0000", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481117+0000", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000003e-07 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481120+0000", 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-21T22:52:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481131+0000", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1242e-05 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481140+0000", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8470000000000002e-06 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583255+0000", 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102114893 2024-09-21T22:52:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583378+0000", 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000123168 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583492+0000", 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011394400000000001 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:37766 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 612352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.66442172000001, 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102768895, 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37766 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481151+0000", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481152+0000", 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5700000000000002e-07 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481155+0000", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1360000000000001e-06 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481166+0000", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1372999999999999e-05 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.481175+0000", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7830000000000004e-06 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583568+0000", 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.102393068 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583754+0000", 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018560000000000001 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.583920+0000", 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016627800000000001 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:38101 3.f 3:f639672f:::rbd_data.111ebb10c685.0000000000000000:e [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.64641514500001, 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096949164000000004, 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 38101 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499158+0000", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499159+0000", 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.17e-06 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499164+0000", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499178+0000", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3206e-05 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.499187+0000", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2760000000000006e-06 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595823+0000", 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.096636575000000002 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.595966+0000", 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014293699999999999 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:46.596107+0000", 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014025299999999999 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150127 3.11 3:8f3f7c38:::rbd_data.111edf735d75.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.323834994999999, 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.12904363599999999, 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150127 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821738+0000", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.821740+0000", 2024-09-21T22:52:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4140000000000001e-06 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847107+0000", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025366738999999999 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847156+0000", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9066000000000001e-05 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847176+0000", 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9911999999999998e-05 2024-09-21T22:52:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847336+0000", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016053700000000001 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.847679+0000", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034236100000000001 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.860259+0000", 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580739000000001 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.948196+0000", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.087937071000000006 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950662+0000", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024654770000000002 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950717+0000", 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5461999999999999e-05 2024-09-21T22:52:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.950781+0000", 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3857999999999997e-05 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:170334 3.15 3:abfe414e:::rbd_object_map.111e36902eb9:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.622396805999998, 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94124513300000001, 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 170334 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523176+0000", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523178+0000", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523190+0000", 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2169e-05 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523221+0000", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-05 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523235+0000", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4535999999999999e-05 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523381+0000", 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014580999999999999 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:41.523401+0000", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461489+0000", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.93808817700000002 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.461844+0000", 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035435600000000001 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.462315+0000", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000471805 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464110+0000", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001794697 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464286+0000", 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017546900000000001 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464339+0000", 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3224999999999999e-05 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:42.464421+0000", 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2106000000000003e-05 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.237 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:31.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T22:52:31.285 INFO:teuthology.orchestra.run.smithi094.stderr:marked in osd.2. 2024-09-21T22:52:31.286 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-21T22:52:31.317 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-21T22:52:31.317 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T22:52:31.317 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-21T22:52:31.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-21T22:52:31.368 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-21T22:52:31.368 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:31.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T22:52:31.499 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-21T22:52:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-21T22:52:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-21T22:52:31.499 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:31.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13520 3.10 3:0991511d:::rbd_data.111e4c6c7d62.00000000000001e0:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.611841583, 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132015110000001, 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13520 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019335+0000", 2024-09-21T22:52:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019337+0000", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.322e-06 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019345+0000", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0849999999999997e-06 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019363+0000", 2024-09-21T22:52:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8651000000000002e-05 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019372+0000", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5769999999999994e-06 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019446+0000", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3566999999999995e-05 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.019454+0000", 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0989999999999999e-06 2024-09-21T22:52:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026930+0000", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074762780000000003 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026942+0000", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.202e-05 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032311+0000", 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0053687149999999 2024-09-21T22:52:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.032537+0000", 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000226197 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13528 3.10 3:0fa48f52:::rbd_data.111e4c6c7d62.00000000000001e8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.607604014, 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0123500439999999, 2024-09-21T22:52:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13528 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023573+0000", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023574+0000", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1009999999999999e-06 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023580+0000", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4589999999999997e-06 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023596+0000", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6013999999999999e-05 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023604+0000", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023673+0000", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8448999999999995e-05 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023679+0000", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1709999999999999e-06 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026974+0000", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032953879999999998 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.026983+0000", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5569999999999996e-06 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033465+0000", 2024-09-21T22:52:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.006481618 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033577+0000", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112568 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.033685+0000", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108043 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035061+0000", 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013759320000000001 2024-09-21T22:52:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035848+0000", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078721399999999995 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035889+0000", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0606999999999999e-05 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035923+0000", 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4113999999999999e-05 2024-09-21T22:52:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:13529 3.10 3:0bd753cf:::rbd_data.111e4c6c7d62.00000000000001e9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.607409116, 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0132335269999999, 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 13529 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023768+0000", 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023769+0000", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3200000000000003e-07 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023773+0000", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023789+0000", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5599000000000001e-05 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023797+0000", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5449999999999998e-06 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023859+0000", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2373999999999996e-05 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.023864+0000", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5369999999999997e-06 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027299+0000", 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034344000000000002 2024-09-21T22:52:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:36.027310+0000", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1409999999999999e-05 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034054+0000", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0067439389999999 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034186+0000", 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013184000000000001 2024-09-21T22:52:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.034268+0000", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1977999999999993e-05 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.035240+0000", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00097210999999999999 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036663+0000", 2024-09-21T22:52:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014225729999999999 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.036873+0000", 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000210485 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:50:37.037001+0000", 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012818899999999999 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150023 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885734486999993, 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067176642999999994, 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150023 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745442+0000", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745443+0000", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7000000000000004e-07 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745456+0000", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2830000000000001e-05 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745467+0000", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0637e-05 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745474+0000", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2230000000000001e-06 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812398+0000", 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066924058999999994 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812494+0000", 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5894e-05 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812619+0000", 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012532999999999999 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150024 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6815744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885692606999996, 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067418757999999995, 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150024 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745484+0000", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745485+0000", 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9999999999999996e-07 2024-09-21T22:52:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745488+0000", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1389999999999999e-06 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745498+0000", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4569999999999994e-06 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745509+0000", 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0893000000000001e-05 2024-09-21T22:52:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812681+0000", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067172154999999997 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812780+0000", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9188e-05 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812903+0000", 2024-09-21T22:52:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012302600000000001 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150025 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6881280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885657021, 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067660329000000005, 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150025 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745520+0000", 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745521+0000", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4300000000000002e-07 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745526+0000", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9629999999999997e-06 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745536+0000", 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8030000000000008e-06 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745543+0000", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1799999999999999e-06 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.812963+0000", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067420333999999998 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813060+0000", 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6484999999999995e-05 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813180+0000", 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120721 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150026 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 6979584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885624839000002, 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067907908000000003, 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150026 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745552+0000", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745553+0000", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1699999999999998e-07 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745556+0000", 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1030000000000002e-06 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745565+0000", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.55e-06 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745572+0000", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7939999999999997e-06 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813242+0000", 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067669335999999997 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813338+0000", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6682999999999996e-05 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813460+0000", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012182500000000001 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150027 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885592813000002, 2024-09-21T22:52:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068176126000000004, 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150027 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745584+0000", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745585+0000", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.64e-07 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745590+0000", 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9230000000000001e-06 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745607+0000", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6682999999999999e-05 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745619+0000", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2607e-05 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813522+0000", 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067902220999999999 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813615+0000", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3325999999999994e-05 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813760+0000", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014540199999999999 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150028 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7208960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885522800999993, 2024-09-21T22:52:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068393153999999998, 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150028 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745654+0000", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745655+0000", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.037e-06 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745662+0000", 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9700000000000002e-06 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745682+0000", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.944e-05 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745695+0000", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3039e-05 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813828+0000", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068133280000000004 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.813927+0000", 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9197999999999995e-05 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814047+0000", 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012019 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150029 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7274496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885458921999998, 2024-09-21T22:52:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068616263999999996, 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150029 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745718+0000", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745719+0000", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0999999999999997e-07 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745725+0000", 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6439999999999997e-06 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745743+0000", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8045000000000001e-05 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745755+0000", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2680000000000001e-05 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814115+0000", 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068359610000000001 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814209+0000", 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.446e-05 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814334+0000", 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012491499999999999 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150030 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7372800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885405679000002, 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068843372999999999, 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150030 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745771+0000", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745772+0000", 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9999999999999995e-07 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745778+0000", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745794+0000", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6331999999999999e-05 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745807+0000", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3134e-05 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814396+0000", 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068589320999999995 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814494+0000", 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7750000000000004e-05 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814615+0000", 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012040300000000001 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150031 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7438336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885354370000002, 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069074204, 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150031 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745823+0000", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4600000000000003e-07 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745829+0000", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745845+0000", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6215999999999999e-05 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745859+0000", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.431e-05 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814680+0000", 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.068821390999999996 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814776+0000", 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5926000000000003e-05 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814897+0000", 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150032 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885303544999999, 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069301728000000007, 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150032 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745873+0000", 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745874+0000", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3399999999999998e-07 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745879+0000", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745894+0000", 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4959e-05 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745906+0000", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.148e-05 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.814958+0000", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069052526000000003 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815056+0000", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7298000000000003e-05 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815175+0000", 2024-09-21T22:52:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119628 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150033 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7667712~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885256385999995, 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069538364000000005, 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150033 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745921+0000", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0699999999999996e-07 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745925+0000", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8700000000000002e-06 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745936+0000", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745943+0000", 2024-09-21T22:52:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4000000000000003e-06 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815235+0000", 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069292261999999993 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815333+0000", 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7554999999999997e-05 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815459+0000", 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012621099999999999 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150034 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7766016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885220849999996, 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069782354000000005, 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150034 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745956+0000", 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745957+0000", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745963+0000", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745978+0000", 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4803e-05 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.745987+0000", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4930000000000004e-06 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815519+0000", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069531246000000005 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815619+0000", 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100243 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815738+0000", 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966199999999999 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150035 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7831552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885172620999995, 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.07001098, 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150035 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746004+0000", 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746005+0000", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6700000000000002e-07 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746021+0000", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6166000000000001e-05 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746042+0000", 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0273000000000001e-05 2024-09-21T22:52:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746055+0000", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815798+0000", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069743515000000006 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.815895+0000", 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7174000000000002e-05 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816015+0000", 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012012500000000001 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150036 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885104479000006, 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070221096999999996, 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150036 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746072+0000", 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746074+0000", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.124e-06 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746082+0000", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746101+0000", 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8102999999999999e-05 2024-09-21T22:52:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746113+0000", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2537e-05 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816074+0000", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.069960594000000001 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816169+0000", 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5502999999999997e-05 2024-09-21T22:52:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816294+0000", 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012442700000000001 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150037 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.885043343000007, 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070439214, 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150037 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746134+0000", 2024-09-21T22:52:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746135+0000", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746141+0000", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0129999999999999e-06 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746158+0000", 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7258999999999999e-05 2024-09-21T22:52:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746174+0000", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5704000000000001e-05 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816355+0000", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070181199 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816453+0000", 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7893999999999995e-05 2024-09-21T22:52:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816573+0000", 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120115 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150038 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8159232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.884985174999997, 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070660555, 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150038 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746192+0000", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746193+0000", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1999999999999998e-07 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746199+0000", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8000000000000004e-06 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746217+0000", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8243000000000001e-05 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746230+0000", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3563e-05 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816635+0000", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070404337999999997 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816732+0000", 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.768e-05 2024-09-21T22:52:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816852+0000", 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120011 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4382.0:150039 3.10 3:0fff88b1:::rbd_data.111e4671229f.0000000000000005:1e [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 68.884929025999995, 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070858076000000006, 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "continuous": false, 2024-09-21T22:52:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4382", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 150039 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746248+0000", 2024-09-21T22:52:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746249+0000", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0300000000000001e-06 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746254+0000", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746272+0000", 2024-09-21T22:52:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7954000000000001e-05 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.746285+0000", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3346000000000001e-05 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816892+0000", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.070606239000000001 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.816985+0000", 2024-09-21T22:52:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3684000000000006e-05 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-21T22:51:22.817106+0000", 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012076 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T22:52:31.722 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T22:52:31.723 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-21T22:52:31.660+0000 7fe4f0d43640 -1 mon.a@0(leader).osd e140 definitely_dead 0 2024-09-21T22:52:31.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T22:52:31.846 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-21T22:52:31.846 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:31.859 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T22:52:31.978 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:31.978 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-21T22:52:31.978 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-21T22:52:31.978 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-21T22:52:31.979 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:31.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T22:52:32.115 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:32.115 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:170372 3.1f 3:f9dd9169:::rbd_data.111e470eb4b3.000000000000000f:head [stat,delete] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 49.607745905000002, 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0073605629999999997, 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 170372 2024-09-21T22:52:32.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504678+0000", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504680+0000", 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7799999999999999e-06 2024-09-21T22:52:32.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504686+0000", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504698+0000", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2033e-05 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504709+0000", 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0943e-05 2024-09-21T22:52:32.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504801+0000", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1952000000000003e-05 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for missing object", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.504985+0000", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000183616 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507615+0000", 2024-09-21T22:52:32.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026300609999999999 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for rw locks", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.507745+0000", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129992 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.508989+0000", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001244235 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509079+0000", 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9844000000000002e-05 2024-09-21T22:52:32.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.509314+0000", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023526999999999999 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511233+0000", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001918955 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511899+0000", 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000666046 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.511950+0000", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0645e-05 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:42.512039+0000", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9409999999999999e-05 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:171075 3.1f 3:f9497c16:::rbd_data.111e470eb4b3.0000000000000006:head [truncate 2359296] snapc 25=[25] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:32.122 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 48.581893774000001, 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0067210710000000003, 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 171075 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530531+0000", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4700000000000001e-07 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530535+0000", 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.002e-06 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530543+0000", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1720000000000007e-06 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.530550+0000", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3709999999999996e-06 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532920+0000", 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002369716 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.532998+0000", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8114000000000005e-05 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.533114+0000", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116049 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.536345+0000", 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032316049999999998 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537163+0000", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081771999999999997 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537207+0000", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4001000000000001e-05 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:43.537252+0000", 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4573999999999997e-05 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:172919 3.2 3:40ea79c0:::rbd_data.111ea9c8c3ca.0000000000000058:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.496516159999999, 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060113579999999996, 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 172919 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615908+0000", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615909+0000", 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615914+0000", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6530000000000002e-06 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615927+0000", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.275e-05 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615934+0000", 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9149999999999996e-06 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.615992+0000", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7655999999999998e-05 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616133+0000", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000140779 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.616249+0000", 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116363 2024-09-21T22:52:32.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.617056+0000", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080715500000000005 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621837+0000", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047805139999999996 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621872+0000", 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5193e-05 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:51.621920+0000", 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7429999999999998e-05 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4382.0:173080 3.19 e101/90 3:9bfb520d:::rbd_data.111edf735d75.0000000000000000:head v 101'475, mlcod=96'473)", 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.493823720999998, 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065553670000000003, 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618601+0000", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618603+0000", 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0420000000000001e-06 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618614+0000", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1664999999999999e-05 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618638+0000", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3275e-05 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618649+0000", 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1744e-05 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618722+0000", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2713999999999995e-05 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.618746+0000", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4405000000000001e-05 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625111+0000", 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063640830000000004 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:52.625156+0000", 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5439000000000003e-05 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.1.0:1748 3.19 e105/90 3:9c82044c:::rbd_data.111e6870f364.00000000000001dd:1b v 105'477, mlcod=98'474)", 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.073085139, 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0080063819999999994, 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039339+0000", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039340+0000", 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.15e-06 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039353+0000", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.305e-05 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039381+0000", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7929e-05 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039402+0000", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0849999999999999e-05 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039553+0000", 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000150622 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.039610+0000", 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7176999999999998e-05 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047239+0000", 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076286549999999998 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.047346+0000", 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106949 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3445 3.15 e105/90 3:af8db21c:::rbd_data.111e6870f364.00000000000001cb:1b v 105'418, mlcod=100'414)", 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.062241167000003, 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0082047070000000003, 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.141 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050183+0000", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050184+0000", 2024-09-21T22:52:32.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5000000000000001e-07 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050194+0000", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9729999999999992e-06 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050220+0000", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6033e-05 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.050238+0000", 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8082999999999999e-05 2024-09-21T22:52:32.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051755+0000", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015167119999999999 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051811+0000", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5693999999999999e-05 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058253+0000", 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0064424069999999998 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.058388+0000", 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000134855 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3446 3.15 e105/90 3:af5dc254:::rbd_data.111e6870f364.0000000000000307:1b v 105'420, mlcod=100'414)", 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.060735555000001, 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0076995509999999998, 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051689+0000", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051690+0000", 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.533e-06 2024-09-21T22:52:32.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051705+0000", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4357e-05 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051733+0000", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8449000000000002e-05 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.051751+0000", 2024-09-21T22:52:32.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8034e-05 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054423+0000", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026720939999999999 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.054483+0000", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9889999999999997e-05 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059283+0000", 2024-09-21T22:52:32.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047999649999999998 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.059388+0000", 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010522900000000001 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.5.0:3022 3.5 e105/93 3:a186aeb4:::rbd_data.111e6870f364.0000000000000063:1b v 105'542, mlcod=100'537)", 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.048031678999997, 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0072172449999999997, 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064393+0000", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064397+0000", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.216e-06 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064459+0000", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2030000000000001e-05 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064481+0000", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2126000000000001e-05 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.064496+0000", 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4534e-05 2024-09-21T22:52:32.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065626+0000", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011300080000000001 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.065679+0000", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3603000000000002e-05 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071492+0000", 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058126189999999998 2024-09-21T22:52:32.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:51:56.071610+0000", 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118109 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3512 3.15 e109/90 3:acd237f2:::rbd_data.111e28e769d9.0000000000000003:17 v 109'429, mlcod=108'426)", 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.048756982, 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098050440000000006, 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063667+0000", 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063668+0000", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7199999999999999e-07 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063678+0000", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7429999999999997e-06 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063706+0000", 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8666e-05 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063725+0000", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8935000000000001e-05 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063789+0000", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3849999999999993e-05 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.063811+0000", 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2161e-05 2024-09-21T22:52:32.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073422+0000", 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096102670000000005 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.073472+0000", 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0850000000000003e-05 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.157 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3513 3.15 e109/90 3:ae06b805:::rbd_data.111e28e769d9.0000000000000000:17 v 109'431, mlcod=108'426)", 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.047354532999996, 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.010242922, 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065070+0000", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065071+0000", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065083+0000", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1219999999999999e-05 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065107+0000", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4246000000000001e-05 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.065121+0000", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4521999999999999e-05 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067286+0000", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002164633 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.067314+0000", 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8036000000000001e-05 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075225+0000", 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079108849999999994 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:00.075313+0000", 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7780000000000003e-05 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174541 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.065743168000001, 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012035209999999999, 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174541 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046681+0000", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046684+0000", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.163e-06 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046701+0000", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6379999999999999e-05 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046733+0000", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2580000000000003e-05 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046756+0000", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2289e-05 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.046945+0000", 2024-09-21T22:52:32.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000189259 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.047997+0000", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001052173 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.049497+0000", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001499656 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.054058+0000", 2024-09-21T22:52:32.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045615569999999999 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058392+0000", 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043334389999999997 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058595+0000", 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020378200000000001 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.058716+0000", 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000120932 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:174547 3.1e 3:7a787065:::rbd_header.111ec49ace6d:head [watch unwatch cookie 140232628726112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.041568706, 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061692500000000003, 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 174547 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070856+0000", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070857+0000", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.497e-06 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070865+0000", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9929999999999995e-06 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070890+0000", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4544000000000001e-05 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.070911+0000", 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1710000000000001e-05 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071641+0000", 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00072938799999999998 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.071813+0000", 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000172595 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.072182+0000", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036826099999999999 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.074511+0000", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023291560000000002 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076667+0000", 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021564980000000002 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.076760+0000", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3012999999999999e-05 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:01.077025+0000", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026459500000000001 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:175724 3.7 3:e7bb7df3:::rbd_data.111e4aac08c7.0000000000000038:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.84278617, 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074439659999999998, 2024-09-21T22:52:32.172 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 175724 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269638+0000", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269639+0000", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269645+0000", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9359999999999996e-06 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269661+0000", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6276000000000001e-05 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269672+0000", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1069e-05 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269816+0000", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014353900000000001 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269908+0000", 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1812999999999997e-05 2024-09-21T22:52:32.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.269999+0000", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1216000000000002e-05 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.270666+0000", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066698500000000004 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.276974+0000", 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063083549999999999 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277035+0000", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0924999999999997e-05 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:09.277082+0000", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6801999999999997e-05 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(osd.3.0:378 3.2 e118/93 ondisk, result = 0)", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:32.177 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.973211750000001, 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071321620000000001, 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139213+0000", 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139215+0000", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3860000000000001e-06 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139227+0000", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1924e-05 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139258+0000", 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0766000000000003e-05 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.139277+0000", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9106e-05 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146285+0000", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070080059999999998 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146304+0000", 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8974000000000002e-05 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:10.146345+0000", 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1e-05 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177349 3.7 3:e2fc42f5:::rbd_header.111e7572a6e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.937677295, 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.018545189, 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177349 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174747+0000", 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174749+0000", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.838e-06 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174756+0000", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5519999999999999e-06 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174771+0000", 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4527e-05 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174781+0000", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0481e-05 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.174878+0000", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6747000000000001e-05 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175063+0000", 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000184527 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.175876+0000", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081354400000000003 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.177282+0000", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405852 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193173+0000", 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015890609999999999 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193241+0000", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8131000000000002e-05 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:19.193292+0000", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1379999999999999e-05 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(osd.7.0:3809 3.15 e134/90 3:aaad2694:::rbd_data.111ecbf88771.0000000000000031:4 v 134'494, mlcod=128'490)", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:32.186 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.8759622900000004, 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0066034850000000001, 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236462+0000", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.65e-07 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236469+0000", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7129999999999999e-06 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236480+0000", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1352000000000001e-05 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.236492+0000", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1411e-05 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.237261+0000", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076960699999999995 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.237298+0000", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6059000000000002e-05 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.241713+0000", 2024-09-21T22:52:32.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044153819999999998 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.243066+0000", 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001352596 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177647 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.8727066260000003, 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094287539999999993, 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177647 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239718+0000", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239719+0000", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7400000000000001e-06 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239726+0000", 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7819999999999996e-06 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239740+0000", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.418e-05 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239749+0000", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.799e-06 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239818+0000", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8758999999999997e-05 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.239911+0000", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3158000000000005e-05 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.240242+0000", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00033125800000000002 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.241890+0000", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016481079999999999 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249043+0000", 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.007152968 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249104+0000", 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0256e-05 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.249146+0000", 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2746e-05 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177651 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.8540581349999998, 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0071939489999999998, 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177651 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258366+0000", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258368+0000", 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.001e-06 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258376+0000", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5499999999999997e-06 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258393+0000", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6813e-05 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258401+0000", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5709999999999998e-06 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258464+0000", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2995000000000002e-05 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258572+0000", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107885 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.258766+0000", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000194422 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.259739+0000", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000972503 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265430+0000", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005691008 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265503+0000", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3322e-05 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:25.265560+0000", 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6879000000000002e-05 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177668 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [watch watch cookie 140232628694864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.4203252380000002, 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0078664040000000005, 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177668 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692099+0000", 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692100+0000", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4600000000000004e-07 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692103+0000", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6339999999999999e-06 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692114+0000", 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0139999999999999e-05 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692119+0000", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1800000000000004e-06 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692180+0000", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0999999999999999e-05 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692275+0000", 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5364000000000004e-05 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.692472+0000", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000197023 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.693277+0000", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080479599999999998 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699813+0000", 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065361899999999999 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699860+0000", 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7301e-05 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.699966+0000", 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010513 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4382.0:177703 3.1e 3:7b053a02:::rbd_header.111ecbf88771:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T22:52:32.206 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.3647038839999999, 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0061480559999999998, 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4382", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2343181602", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 177703 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747720+0000", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747722+0000", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0750000000000001e-06 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747727+0000", 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4090000000000002e-06 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747742+0000", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4973e-05 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747751+0000", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7239999999999998e-06 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747788+0000", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7267000000000001e-05 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-21T22:52:32.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747834+0000", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6124000000000001e-05 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.747938+0000", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103624 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.748474+0000", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053669600000000003 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753809+0000", 2024-09-21T22:52:32.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0053350849999999998 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753842+0000", 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2271999999999999e-05 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-21T22:52:27.753869+0000", 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6806999999999999e-05 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-21T22:52:32.211 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:marked in osd.4. 2024-09-21T22:52:32.322 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-21T22:52:32.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T22:52:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a: {} 2024-09-21T22:52:32.450 INFO:teuthology.orchestra.run.smithi094.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:52:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b: {} 2024-09-21T22:52:32.464 INFO:teuthology.orchestra.run.smithi094.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:52:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c: {} 2024-09-21T22:52:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T22:52:32.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T22:52:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T22:52:33.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T22:52:34.310 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T22:52:34.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T22:52:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T22:52:35.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T22:52:36.047 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T22:52:36.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T22:52:37.049 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T22:52:37.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T22:52:38.058 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T22:52:38.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T22:52:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T22:52:39.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T22:52:40.074 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T22:52:40.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T22:52:41.056 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T22:52:41.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T22:52:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T22:52:42.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T22:52:43.350 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T22:52:43.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-21T22:52:44.359 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T22:52:44.381 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T22:52:45.356 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T22:52:45.380 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T22:52:46.372 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T22:52:46.396 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T22:52:47.381 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T22:52:47.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T22:52:48.389 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T22:52:48.413 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T22:52:48.413 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T22:52:48.414 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T22:52:48.414 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T22:52:48.414 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T22:52:48.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:52:48.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:48.715 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":157,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:52:48.381852+0000","last_up_change":"2024-09-21T22:51:41.430550+0000","last_in_change":"2024-09-21T22:52:32.286494+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T22:49:55.226638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T22:50:05.084190+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":77,"up_from":90,"up_thru":134,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6801","nonce":2575188997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6803","nonce":2575188997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6807","nonce":2575188997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6805","nonce":2575188997}]},"public_addr":"172.21.15.94:6801/2575188997","cluster_addr":"172.21.15.94:6803/2575188997","heartbeat_back_addr":"172.21.15.94:6807/2575188997","heartbeat_front_addr":"172.21.15.94:6805/2575188997","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":42,"up_thru":147,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6811","nonce":3470582226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6813","nonce":3470582226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6817","nonce":3470582226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6815","nonce":3470582226}]},"public_addr":"172.21.15.99:6811/3470582226","cluster_addr":"172.21.15.99:6813/3470582226","heartbeat_back_addr":"172.21.15.99:6817/3470582226","heartbeat_front_addr":"172.21.15.99:6815/3470582226","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":24},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1f","osds":[3,5]}],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:52:48.733 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T22:52:48.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:52:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:49.038 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":157,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:52:48.381852+0000","last_up_change":"2024-09-21T22:51:41.430550+0000","last_in_change":"2024-09-21T22:52:32.286494+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T22:49:55.226638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T22:50:05.084190+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":77,"up_from":90,"up_thru":134,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6801","nonce":2575188997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6803","nonce":2575188997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6807","nonce":2575188997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6805","nonce":2575188997}]},"public_addr":"172.21.15.94:6801/2575188997","cluster_addr":"172.21.15.94:6803/2575188997","heartbeat_back_addr":"172.21.15.94:6807/2575188997","heartbeat_front_addr":"172.21.15.94:6805/2575188997","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":42,"up_thru":147,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6811","nonce":3470582226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6813","nonce":3470582226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6817","nonce":3470582226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6815","nonce":3470582226}]},"public_addr":"172.21.15.99:6811/3470582226","cluster_addr":"172.21.15.99:6813/3470582226","heartbeat_back_addr":"172.21.15.99:6817/3470582226","heartbeat_front_addr":"172.21.15.99:6815/3470582226","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":24},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1f","osds":[3,5]}],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:52:49.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T22:52:49.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T22:52:49.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T22:52:49.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T22:52:49.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T22:52:49.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T22:52:49.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T22:52:49.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T22:52:49.321 INFO:teuthology.orchestra.run.smithi094.stdout:64424509479 2024-09-21T22:52:49.321 INFO:teuthology.orchestra.run.smithi094.stdout:64424509479 2024-09-21T22:52:49.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T22:52:49.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T22:52:49.323 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T22:52:49.326 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T22:52:49.328 INFO:teuthology.orchestra.run.smithi094.stdout:68719476775 2024-09-21T22:52:49.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T22:52:49.329 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T22:52:49.330 INFO:teuthology.orchestra.run.smithi094.stdout:386547056655 2024-09-21T22:52:49.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T22:52:49.334 INFO:teuthology.orchestra.run.smithi094.stdout:180388626461 2024-09-21T22:52:49.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T22:52:49.818 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.841 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509480 for osd.1 2024-09-21T22:52:49.842 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.854 INFO:teuthology.orchestra.run.smithi094.stdout:64424509479 2024-09-21T22:52:49.859 INFO:teuthology.orchestra.run.smithi094.stdout:64424509479 2024-09-21T22:52:49.865 INFO:teuthology.orchestra.run.smithi094.stdout:68719476775 2024-09-21T22:52:49.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509480 for osd.4 2024-09-21T22:52:49.867 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.878 INFO:teuthology.orchestra.run.smithi094.stdout:180388626461 2024-09-21T22:52:49.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.0 2024-09-21T22:52:49.885 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.2 2024-09-21T22:52:49.891 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.893 INFO:teuthology.orchestra.run.smithi094.stdout:386547056655 2024-09-21T22:52:49.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476775 got 68719476775 for osd.5 2024-09-21T22:52:49.898 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.900 INFO:teuthology.orchestra.run.smithi094.stdout:64424509480 2024-09-21T22:52:49.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 180388626461 got 180388626461 for osd.6 2024-09-21T22:52:49.904 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 386547056655 got 386547056655 for osd.3 2024-09-21T22:52:49.914 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509480 for osd.7 2024-09-21T22:52:49.919 DEBUG:teuthology.parallel:result is None 2024-09-21T22:52:49.919 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T22:52:49.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:50.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:50.238 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:50.254 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-21T22:52:49.318486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411688,"kb_used_data":15792,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563032,"statfs":{"total":773094113280,"available":772672544768,"internally_reserved":0,"allocated":16171008,"data_stored":8336306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001119"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8155,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067136+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:48.067136+0000","last_peered":"2024-09-21T22:52:48.067136+0000","last_clean":"2024-09-21T22:52:48.067136+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:48.067136+0000","last_undegraded":"2024-09-21T22:52:48.067136+0000","last_fullsized":"2024-09-21T22:52:48.067136+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3056,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066965+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:48.066965+0000","last_peered":"2024-09-21T22:52:48.066965+0000","last_clean":"2024-09-21T22:52:48.066965+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:48.066965+0000","last_undegraded":"2024-09-21T22:52:48.066965+0000","last_fullsized":"2024-09-21T22:52:48.066965+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7369,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066836+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:48.066836+0000","last_peered":"2024-09-21T22:52:48.066836+0000","last_clean":"2024-09-21T22:52:48.066836+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:48.066836+0000","last_undegraded":"2024-09-21T22:52:48.066836+0000","last_fullsized":"2024-09-21T22:52:48.066836+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9390,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3511,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066985+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:48.066985+0000","last_peered":"2024-09-21T22:52:48.066985+0000","last_clean":"2024-09-21T22:52:48.066985+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:48.066985+0000","last_undegraded":"2024-09-21T22:52:48.066985+0000","last_fullsized":"2024-09-21T22:52:48.066985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8753,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067097+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:48.067097+0000","last_peered":"2024-09-21T22:52:48.067097+0000","last_clean":"2024-09-21T22:52:48.067097+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:48.067097+0000","last_undegraded":"2024-09-21T22:52:48.067097+0000","last_fullsized":"2024-09-21T22:52:48.067097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11522,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066438+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:48.066438+0000","last_peered":"2024-09-21T22:52:48.066438+0000","last_clean":"2024-09-21T22:52:48.066438+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:48.066438+0000","last_undegraded":"2024-09-21T22:52:48.066438+0000","last_fullsized":"2024-09-21T22:52:48.066438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":252,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066972+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:48.066972+0000","last_peered":"2024-09-21T22:52:48.066972+0000","last_clean":"2024-09-21T22:52:48.066972+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:48.066972+0000","last_undegraded":"2024-09-21T22:52:48.066972+0000","last_fullsized":"2024-09-21T22:52:48.066972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6766,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066645+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:48.066645+0000","last_peered":"2024-09-21T22:52:48.066645+0000","last_clean":"2024-09-21T22:52:48.066645+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:48.066645+0000","last_undegraded":"2024-09-21T22:52:48.066645+0000","last_fullsized":"2024-09-21T22:52:48.066645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":291,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066497+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:48.066497+0000","last_peered":"2024-09-21T22:52:48.066497+0000","last_clean":"2024-09-21T22:52:48.066497+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:48.066497+0000","last_undegraded":"2024-09-21T22:52:48.066497+0000","last_fullsized":"2024-09-21T22:52:48.066497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066771+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:48.066771+0000","last_peered":"2024-09-21T22:52:48.066771+0000","last_clean":"2024-09-21T22:52:48.066771+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:48.066771+0000","last_undegraded":"2024-09-21T22:52:48.066771+0000","last_fullsized":"2024-09-21T22:52:48.066771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3453,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066186+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:48.066186+0000","last_peered":"2024-09-21T22:52:48.066186+0000","last_clean":"2024-09-21T22:52:48.066186+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:48.066186+0000","last_undegraded":"2024-09-21T22:52:48.066186+0000","last_fullsized":"2024-09-21T22:52:48.066186+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":284,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066358+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:48.066358+0000","last_peered":"2024-09-21T22:52:48.066358+0000","last_clean":"2024-09-21T22:52:48.066358+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:48.066358+0000","last_undegraded":"2024-09-21T22:52:48.066358+0000","last_fullsized":"2024-09-21T22:52:48.066358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2808,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067203+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:48.067203+0000","last_peered":"2024-09-21T22:52:48.067203+0000","last_clean":"2024-09-21T22:52:48.067203+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:48.067203+0000","last_undegraded":"2024-09-21T22:52:48.067203+0000","last_fullsized":"2024-09-21T22:52:48.067203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":287,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066442+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:48.066442+0000","last_peered":"2024-09-21T22:52:48.066442+0000","last_clean":"2024-09-21T22:52:48.066442+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:48.066442+0000","last_undegraded":"2024-09-21T22:52:48.066442+0000","last_fullsized":"2024-09-21T22:52:48.066442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5571,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6400,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066547+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:48.066547+0000","last_peered":"2024-09-21T22:52:48.066547+0000","last_clean":"2024-09-21T22:52:48.066547+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:48.066547+0000","last_undegraded":"2024-09-21T22:52:48.066547+0000","last_fullsized":"2024-09-21T22:52:48.066547+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065792+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:48.065792+0000","last_peered":"2024-09-21T22:52:48.065792+0000","last_clean":"2024-09-21T22:52:48.065792+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:48.065792+0000","last_undegraded":"2024-09-21T22:52:48.065792+0000","last_fullsized":"2024-09-21T22:52:48.065792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066559+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:48.066559+0000","last_peered":"2024-09-21T22:52:48.066559+0000","last_clean":"2024-09-21T22:52:48.066559+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:48.066559+0000","last_undegraded":"2024-09-21T22:52:48.066559+0000","last_fullsized":"2024-09-21T22:52:48.066559+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":436,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066063+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:48.066063+0000","last_peered":"2024-09-21T22:52:48.066063+0000","last_clean":"2024-09-21T22:52:48.066063+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:48.066063+0000","last_undegraded":"2024-09-21T22:52:48.066063+0000","last_fullsized":"2024-09-21T22:52:48.066063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066073+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:48.066073+0000","last_peered":"2024-09-21T22:52:48.066073+0000","last_clean":"2024-09-21T22:52:48.066073+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:48.066073+0000","last_undegraded":"2024-09-21T22:52:48.066073+0000","last_fullsized":"2024-09-21T22:52:48.066073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2049,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066678+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:48.066678+0000","last_peered":"2024-09-21T22:52:48.066678+0000","last_clean":"2024-09-21T22:52:48.066678+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:48.066678+0000","last_undegraded":"2024-09-21T22:52:48.066678+0000","last_fullsized":"2024-09-21T22:52:48.066678+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067260+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:48.067260+0000","last_peered":"2024-09-21T22:52:48.067260+0000","last_clean":"2024-09-21T22:52:48.067260+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:48.067260+0000","last_undegraded":"2024-09-21T22:52:48.067260+0000","last_fullsized":"2024-09-21T22:52:48.067260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4068,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066647+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:48.066647+0000","last_peered":"2024-09-21T22:52:48.066647+0000","last_clean":"2024-09-21T22:52:48.066647+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:48.066647+0000","last_undegraded":"2024-09-21T22:52:48.066647+0000","last_fullsized":"2024-09-21T22:52:48.066647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7326,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067139+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:48.067139+0000","last_peered":"2024-09-21T22:52:48.067139+0000","last_clean":"2024-09-21T22:52:48.067139+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:48.067139+0000","last_undegraded":"2024-09-21T22:52:48.067139+0000","last_fullsized":"2024-09-21T22:52:48.067139+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5955,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066272+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:48.066272+0000","last_peered":"2024-09-21T22:52:48.066272+0000","last_clean":"2024-09-21T22:52:48.066272+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:48.066272+0000","last_undegraded":"2024-09-21T22:52:48.066272+0000","last_fullsized":"2024-09-21T22:52:48.066272+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7564,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066385+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:48.066385+0000","last_peered":"2024-09-21T22:52:48.066385+0000","last_clean":"2024-09-21T22:52:48.066385+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:48.066385+0000","last_undegraded":"2024-09-21T22:52:48.066385+0000","last_fullsized":"2024-09-21T22:52:48.066385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065804+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:48.065804+0000","last_peered":"2024-09-21T22:52:48.065804+0000","last_clean":"2024-09-21T22:52:48.065804+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:48.065804+0000","last_undegraded":"2024-09-21T22:52:48.065804+0000","last_fullsized":"2024-09-21T22:52:48.065804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8649,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066523+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:48.066523+0000","last_peered":"2024-09-21T22:52:48.066523+0000","last_clean":"2024-09-21T22:52:48.066523+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:48.066523+0000","last_undegraded":"2024-09-21T22:52:48.066523+0000","last_fullsized":"2024-09-21T22:52:48.066523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8908,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065872+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:48.065872+0000","last_peered":"2024-09-21T22:52:48.065872+0000","last_clean":"2024-09-21T22:52:48.065872+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:48.065872+0000","last_undegraded":"2024-09-21T22:52:48.065872+0000","last_fullsized":"2024-09-21T22:52:48.065872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1871872,"data_stored":892988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626461,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":1824,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":893007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056655,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:50.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:50.543 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:50.559 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-21T22:52:49.318486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411688,"kb_used_data":15792,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563032,"statfs":{"total":773094113280,"available":772672544768,"internally_reserved":0,"allocated":16171008,"data_stored":8336306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001119"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8155,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067136+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:48.067136+0000","last_peered":"2024-09-21T22:52:48.067136+0000","last_clean":"2024-09-21T22:52:48.067136+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:48.067136+0000","last_undegraded":"2024-09-21T22:52:48.067136+0000","last_fullsized":"2024-09-21T22:52:48.067136+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3056,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066965+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:48.066965+0000","last_peered":"2024-09-21T22:52:48.066965+0000","last_clean":"2024-09-21T22:52:48.066965+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:48.066965+0000","last_undegraded":"2024-09-21T22:52:48.066965+0000","last_fullsized":"2024-09-21T22:52:48.066965+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7369,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066836+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:48.066836+0000","last_peered":"2024-09-21T22:52:48.066836+0000","last_clean":"2024-09-21T22:52:48.066836+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:48.066836+0000","last_undegraded":"2024-09-21T22:52:48.066836+0000","last_fullsized":"2024-09-21T22:52:48.066836+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9390,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3511,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066985+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:48.066985+0000","last_peered":"2024-09-21T22:52:48.066985+0000","last_clean":"2024-09-21T22:52:48.066985+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:48.066985+0000","last_undegraded":"2024-09-21T22:52:48.066985+0000","last_fullsized":"2024-09-21T22:52:48.066985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8753,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067097+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:48.067097+0000","last_peered":"2024-09-21T22:52:48.067097+0000","last_clean":"2024-09-21T22:52:48.067097+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:48.067097+0000","last_undegraded":"2024-09-21T22:52:48.067097+0000","last_fullsized":"2024-09-21T22:52:48.067097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11522,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066438+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:48.066438+0000","last_peered":"2024-09-21T22:52:48.066438+0000","last_clean":"2024-09-21T22:52:48.066438+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:48.066438+0000","last_undegraded":"2024-09-21T22:52:48.066438+0000","last_fullsized":"2024-09-21T22:52:48.066438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":252,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066972+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:48.066972+0000","last_peered":"2024-09-21T22:52:48.066972+0000","last_clean":"2024-09-21T22:52:48.066972+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:48.066972+0000","last_undegraded":"2024-09-21T22:52:48.066972+0000","last_fullsized":"2024-09-21T22:52:48.066972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6766,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066645+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:48.066645+0000","last_peered":"2024-09-21T22:52:48.066645+0000","last_clean":"2024-09-21T22:52:48.066645+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:48.066645+0000","last_undegraded":"2024-09-21T22:52:48.066645+0000","last_fullsized":"2024-09-21T22:52:48.066645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":291,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066497+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:48.066497+0000","last_peered":"2024-09-21T22:52:48.066497+0000","last_clean":"2024-09-21T22:52:48.066497+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:48.066497+0000","last_undegraded":"2024-09-21T22:52:48.066497+0000","last_fullsized":"2024-09-21T22:52:48.066497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066771+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:48.066771+0000","last_peered":"2024-09-21T22:52:48.066771+0000","last_clean":"2024-09-21T22:52:48.066771+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:48.066771+0000","last_undegraded":"2024-09-21T22:52:48.066771+0000","last_fullsized":"2024-09-21T22:52:48.066771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3453,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066186+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:48.066186+0000","last_peered":"2024-09-21T22:52:48.066186+0000","last_clean":"2024-09-21T22:52:48.066186+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:48.066186+0000","last_undegraded":"2024-09-21T22:52:48.066186+0000","last_fullsized":"2024-09-21T22:52:48.066186+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":284,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066358+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:48.066358+0000","last_peered":"2024-09-21T22:52:48.066358+0000","last_clean":"2024-09-21T22:52:48.066358+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:48.066358+0000","last_undegraded":"2024-09-21T22:52:48.066358+0000","last_fullsized":"2024-09-21T22:52:48.066358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2808,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067203+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:48.067203+0000","last_peered":"2024-09-21T22:52:48.067203+0000","last_clean":"2024-09-21T22:52:48.067203+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:48.067203+0000","last_undegraded":"2024-09-21T22:52:48.067203+0000","last_fullsized":"2024-09-21T22:52:48.067203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":287,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066442+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:48.066442+0000","last_peered":"2024-09-21T22:52:48.066442+0000","last_clean":"2024-09-21T22:52:48.066442+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:48.066442+0000","last_undegraded":"2024-09-21T22:52:48.066442+0000","last_fullsized":"2024-09-21T22:52:48.066442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5571,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6400,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066547+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:48.066547+0000","last_peered":"2024-09-21T22:52:48.066547+0000","last_clean":"2024-09-21T22:52:48.066547+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:48.066547+0000","last_undegraded":"2024-09-21T22:52:48.066547+0000","last_fullsized":"2024-09-21T22:52:48.066547+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065792+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:48.065792+0000","last_peered":"2024-09-21T22:52:48.065792+0000","last_clean":"2024-09-21T22:52:48.065792+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:48.065792+0000","last_undegraded":"2024-09-21T22:52:48.065792+0000","last_fullsized":"2024-09-21T22:52:48.065792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066559+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:48.066559+0000","last_peered":"2024-09-21T22:52:48.066559+0000","last_clean":"2024-09-21T22:52:48.066559+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:48.066559+0000","last_undegraded":"2024-09-21T22:52:48.066559+0000","last_fullsized":"2024-09-21T22:52:48.066559+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":436,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066063+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:48.066063+0000","last_peered":"2024-09-21T22:52:48.066063+0000","last_clean":"2024-09-21T22:52:48.066063+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:48.066063+0000","last_undegraded":"2024-09-21T22:52:48.066063+0000","last_fullsized":"2024-09-21T22:52:48.066063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066073+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:48.066073+0000","last_peered":"2024-09-21T22:52:48.066073+0000","last_clean":"2024-09-21T22:52:48.066073+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:48.066073+0000","last_undegraded":"2024-09-21T22:52:48.066073+0000","last_fullsized":"2024-09-21T22:52:48.066073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2049,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066678+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:48.066678+0000","last_peered":"2024-09-21T22:52:48.066678+0000","last_clean":"2024-09-21T22:52:48.066678+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:48.066678+0000","last_undegraded":"2024-09-21T22:52:48.066678+0000","last_fullsized":"2024-09-21T22:52:48.066678+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067260+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:48.067260+0000","last_peered":"2024-09-21T22:52:48.067260+0000","last_clean":"2024-09-21T22:52:48.067260+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:48.067260+0000","last_undegraded":"2024-09-21T22:52:48.067260+0000","last_fullsized":"2024-09-21T22:52:48.067260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4068,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066647+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:48.066647+0000","last_peered":"2024-09-21T22:52:48.066647+0000","last_clean":"2024-09-21T22:52:48.066647+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:48.066647+0000","last_undegraded":"2024-09-21T22:52:48.066647+0000","last_fullsized":"2024-09-21T22:52:48.066647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7326,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067139+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:48.067139+0000","last_peered":"2024-09-21T22:52:48.067139+0000","last_clean":"2024-09-21T22:52:48.067139+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:48.067139+0000","last_undegraded":"2024-09-21T22:52:48.067139+0000","last_fullsized":"2024-09-21T22:52:48.067139+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5955,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066272+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:48.066272+0000","last_peered":"2024-09-21T22:52:48.066272+0000","last_clean":"2024-09-21T22:52:48.066272+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:48.066272+0000","last_undegraded":"2024-09-21T22:52:48.066272+0000","last_fullsized":"2024-09-21T22:52:48.066272+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7564,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066385+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:48.066385+0000","last_peered":"2024-09-21T22:52:48.066385+0000","last_clean":"2024-09-21T22:52:48.066385+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:48.066385+0000","last_undegraded":"2024-09-21T22:52:48.066385+0000","last_fullsized":"2024-09-21T22:52:48.066385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065804+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:48.065804+0000","last_peered":"2024-09-21T22:52:48.065804+0000","last_clean":"2024-09-21T22:52:48.065804+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:48.065804+0000","last_undegraded":"2024-09-21T22:52:48.065804+0000","last_fullsized":"2024-09-21T22:52:48.065804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8649,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066523+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:48.066523+0000","last_peered":"2024-09-21T22:52:48.066523+0000","last_clean":"2024-09-21T22:52:48.066523+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:48.066523+0000","last_undegraded":"2024-09-21T22:52:48.066523+0000","last_fullsized":"2024-09-21T22:52:48.066523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8908,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065872+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:48.065872+0000","last_peered":"2024-09-21T22:52:48.065872+0000","last_clean":"2024-09-21T22:52:48.065872+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:48.065872+0000","last_undegraded":"2024-09-21T22:52:48.065872+0000","last_fullsized":"2024-09-21T22:52:48.065872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1871872,"data_stored":892988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626461,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":1824,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":893007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056655,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:50.561 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T22:52:50.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T22:52:50.854 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T22:52:50.864 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T22:52:50.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:51.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:51.158 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-21T22:52:49.318486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411688,"kb_used_data":15792,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563032,"statfs":{"total":773094113280,"available":772672544768,"internally_reserved":0,"allocated":16171008,"data_stored":8336306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001119"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8155,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067136+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:48.067136+0000","last_peered":"2024-09-21T22:52:48.067136+0000","last_clean":"2024-09-21T22:52:48.067136+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:48.067136+0000","last_undegraded":"2024-09-21T22:52:48.067136+0000","last_fullsized":"2024-09-21T22:52:48.067136+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3056,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066965+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:48.066965+0000","last_peered":"2024-09-21T22:52:48.066965+0000","last_clean":"2024-09-21T22:52:48.066965+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:48.066965+0000","last_undegraded":"2024-09-21T22:52:48.066965+0000","last_fullsized":"2024-09-21T22:52:48.066965+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7369,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066836+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:48.066836+0000","last_peered":"2024-09-21T22:52:48.066836+0000","last_clean":"2024-09-21T22:52:48.066836+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:48.066836+0000","last_undegraded":"2024-09-21T22:52:48.066836+0000","last_fullsized":"2024-09-21T22:52:48.066836+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9390,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3511,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066985+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:48.066985+0000","last_peered":"2024-09-21T22:52:48.066985+0000","last_clean":"2024-09-21T22:52:48.066985+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:48.066985+0000","last_undegraded":"2024-09-21T22:52:48.066985+0000","last_fullsized":"2024-09-21T22:52:48.066985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8753,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067097+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:48.067097+0000","last_peered":"2024-09-21T22:52:48.067097+0000","last_clean":"2024-09-21T22:52:48.067097+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:48.067097+0000","last_undegraded":"2024-09-21T22:52:48.067097+0000","last_fullsized":"2024-09-21T22:52:48.067097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11522,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066438+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:48.066438+0000","last_peered":"2024-09-21T22:52:48.066438+0000","last_clean":"2024-09-21T22:52:48.066438+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:48.066438+0000","last_undegraded":"2024-09-21T22:52:48.066438+0000","last_fullsized":"2024-09-21T22:52:48.066438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":252,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066972+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:48.066972+0000","last_peered":"2024-09-21T22:52:48.066972+0000","last_clean":"2024-09-21T22:52:48.066972+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:48.066972+0000","last_undegraded":"2024-09-21T22:52:48.066972+0000","last_fullsized":"2024-09-21T22:52:48.066972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6766,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066645+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:48.066645+0000","last_peered":"2024-09-21T22:52:48.066645+0000","last_clean":"2024-09-21T22:52:48.066645+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:48.066645+0000","last_undegraded":"2024-09-21T22:52:48.066645+0000","last_fullsized":"2024-09-21T22:52:48.066645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":291,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066497+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:48.066497+0000","last_peered":"2024-09-21T22:52:48.066497+0000","last_clean":"2024-09-21T22:52:48.066497+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:48.066497+0000","last_undegraded":"2024-09-21T22:52:48.066497+0000","last_fullsized":"2024-09-21T22:52:48.066497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066771+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:48.066771+0000","last_peered":"2024-09-21T22:52:48.066771+0000","last_clean":"2024-09-21T22:52:48.066771+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:48.066771+0000","last_undegraded":"2024-09-21T22:52:48.066771+0000","last_fullsized":"2024-09-21T22:52:48.066771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3453,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066186+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:48.066186+0000","last_peered":"2024-09-21T22:52:48.066186+0000","last_clean":"2024-09-21T22:52:48.066186+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:48.066186+0000","last_undegraded":"2024-09-21T22:52:48.066186+0000","last_fullsized":"2024-09-21T22:52:48.066186+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":284,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066358+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:48.066358+0000","last_peered":"2024-09-21T22:52:48.066358+0000","last_clean":"2024-09-21T22:52:48.066358+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:48.066358+0000","last_undegraded":"2024-09-21T22:52:48.066358+0000","last_fullsized":"2024-09-21T22:52:48.066358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2808,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067203+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:48.067203+0000","last_peered":"2024-09-21T22:52:48.067203+0000","last_clean":"2024-09-21T22:52:48.067203+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:48.067203+0000","last_undegraded":"2024-09-21T22:52:48.067203+0000","last_fullsized":"2024-09-21T22:52:48.067203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":287,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066442+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:48.066442+0000","last_peered":"2024-09-21T22:52:48.066442+0000","last_clean":"2024-09-21T22:52:48.066442+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:48.066442+0000","last_undegraded":"2024-09-21T22:52:48.066442+0000","last_fullsized":"2024-09-21T22:52:48.066442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5571,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065993+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:48.065993+0000","last_peered":"2024-09-21T22:52:48.065993+0000","last_clean":"2024-09-21T22:52:48.065993+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:48.065993+0000","last_undegraded":"2024-09-21T22:52:48.065993+0000","last_fullsized":"2024-09-21T22:52:48.065993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6400,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066547+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:48.066547+0000","last_peered":"2024-09-21T22:52:48.066547+0000","last_clean":"2024-09-21T22:52:48.066547+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:48.066547+0000","last_undegraded":"2024-09-21T22:52:48.066547+0000","last_fullsized":"2024-09-21T22:52:48.066547+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065792+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:48.065792+0000","last_peered":"2024-09-21T22:52:48.065792+0000","last_clean":"2024-09-21T22:52:48.065792+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:48.065792+0000","last_undegraded":"2024-09-21T22:52:48.065792+0000","last_fullsized":"2024-09-21T22:52:48.065792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066559+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:48.066559+0000","last_peered":"2024-09-21T22:52:48.066559+0000","last_clean":"2024-09-21T22:52:48.066559+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:48.066559+0000","last_undegraded":"2024-09-21T22:52:48.066559+0000","last_fullsized":"2024-09-21T22:52:48.066559+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":436,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066063+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:48.066063+0000","last_peered":"2024-09-21T22:52:48.066063+0000","last_clean":"2024-09-21T22:52:48.066063+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:48.066063+0000","last_undegraded":"2024-09-21T22:52:48.066063+0000","last_fullsized":"2024-09-21T22:52:48.066063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066073+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:48.066073+0000","last_peered":"2024-09-21T22:52:48.066073+0000","last_clean":"2024-09-21T22:52:48.066073+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:48.066073+0000","last_undegraded":"2024-09-21T22:52:48.066073+0000","last_fullsized":"2024-09-21T22:52:48.066073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2049,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066678+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:48.066678+0000","last_peered":"2024-09-21T22:52:48.066678+0000","last_clean":"2024-09-21T22:52:48.066678+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:48.066678+0000","last_undegraded":"2024-09-21T22:52:48.066678+0000","last_fullsized":"2024-09-21T22:52:48.066678+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067260+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:48.067260+0000","last_peered":"2024-09-21T22:52:48.067260+0000","last_clean":"2024-09-21T22:52:48.067260+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:48.067260+0000","last_undegraded":"2024-09-21T22:52:48.067260+0000","last_fullsized":"2024-09-21T22:52:48.067260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4068,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066647+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:48.066647+0000","last_peered":"2024-09-21T22:52:48.066647+0000","last_clean":"2024-09-21T22:52:48.066647+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:48.066647+0000","last_undegraded":"2024-09-21T22:52:48.066647+0000","last_fullsized":"2024-09-21T22:52:48.066647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7326,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067139+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:48.067139+0000","last_peered":"2024-09-21T22:52:48.067139+0000","last_clean":"2024-09-21T22:52:48.067139+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:48.067139+0000","last_undegraded":"2024-09-21T22:52:48.067139+0000","last_fullsized":"2024-09-21T22:52:48.067139+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5955,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066272+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:48.066272+0000","last_peered":"2024-09-21T22:52:48.066272+0000","last_clean":"2024-09-21T22:52:48.066272+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:48.066272+0000","last_undegraded":"2024-09-21T22:52:48.066272+0000","last_fullsized":"2024-09-21T22:52:48.066272+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7564,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066385+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:48.066385+0000","last_peered":"2024-09-21T22:52:48.066385+0000","last_clean":"2024-09-21T22:52:48.066385+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:48.066385+0000","last_undegraded":"2024-09-21T22:52:48.066385+0000","last_fullsized":"2024-09-21T22:52:48.066385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065804+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:48.065804+0000","last_peered":"2024-09-21T22:52:48.065804+0000","last_clean":"2024-09-21T22:52:48.065804+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:48.065804+0000","last_undegraded":"2024-09-21T22:52:48.065804+0000","last_fullsized":"2024-09-21T22:52:48.065804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8649,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066523+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:48.066523+0000","last_peered":"2024-09-21T22:52:48.066523+0000","last_clean":"2024-09-21T22:52:48.066523+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:48.066523+0000","last_undegraded":"2024-09-21T22:52:48.066523+0000","last_fullsized":"2024-09-21T22:52:48.066523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8908,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.065872+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:48.065872+0000","last_peered":"2024-09-21T22:52:48.065872+0000","last_clean":"2024-09-21T22:52:48.065872+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:48.065872+0000","last_undegraded":"2024-09-21T22:52:48.065872+0000","last_fullsized":"2024-09-21T22:52:48.065872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1871872,"data_stored":892988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626461,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":1824,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1875968,"data_stored":893007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056655,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:51.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:51.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:51.462 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:51.463 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-21T22:52:51.319268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411708,"kb_used_data":15816,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563012,"statfs":{"total":773094113280,"available":772672524288,"internally_reserved":0,"allocated":16195584,"data_stored":8348048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.962042"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8155,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067136+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:48.067136+0000","last_peered":"2024-09-21T22:52:48.067136+0000","last_clean":"2024-09-21T22:52:48.067136+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:48.067136+0000","last_undegraded":"2024-09-21T22:52:48.067136+0000","last_fullsized":"2024-09-21T22:52:48.067136+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3056,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066965+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:48.066965+0000","last_peered":"2024-09-21T22:52:48.066965+0000","last_clean":"2024-09-21T22:52:48.066965+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:48.066965+0000","last_undegraded":"2024-09-21T22:52:48.066965+0000","last_fullsized":"2024-09-21T22:52:48.066965+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7371,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944885+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:49.944885+0000","last_peered":"2024-09-21T22:52:49.944885+0000","last_clean":"2024-09-21T22:52:49.944885+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:49.944885+0000","last_undegraded":"2024-09-21T22:52:49.944885+0000","last_fullsized":"2024-09-21T22:52:49.944885+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9392,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165149+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:50.165149+0000","last_peered":"2024-09-21T22:52:50.165149+0000","last_clean":"2024-09-21T22:52:50.165149+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:50.165149+0000","last_undegraded":"2024-09-21T22:52:50.165149+0000","last_fullsized":"2024-09-21T22:52:50.165149+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3513,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944896+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:49.944896+0000","last_peered":"2024-09-21T22:52:49.944896+0000","last_clean":"2024-09-21T22:52:49.944896+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:49.944896+0000","last_undegraded":"2024-09-21T22:52:49.944896+0000","last_fullsized":"2024-09-21T22:52:49.944896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8753,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067097+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:48.067097+0000","last_peered":"2024-09-21T22:52:48.067097+0000","last_clean":"2024-09-21T22:52:48.067097+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:48.067097+0000","last_undegraded":"2024-09-21T22:52:48.067097+0000","last_fullsized":"2024-09-21T22:52:48.067097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11524,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812813+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:49.812813+0000","last_peered":"2024-09-21T22:52:49.812813+0000","last_clean":"2024-09-21T22:52:49.812813+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:49.812813+0000","last_undegraded":"2024-09-21T22:52:49.812813+0000","last_fullsized":"2024-09-21T22:52:49.812813+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":252,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066972+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:48.066972+0000","last_peered":"2024-09-21T22:52:48.066972+0000","last_clean":"2024-09-21T22:52:48.066972+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:48.066972+0000","last_undegraded":"2024-09-21T22:52:48.066972+0000","last_fullsized":"2024-09-21T22:52:48.066972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6768,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944888+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:49.944888+0000","last_peered":"2024-09-21T22:52:49.944888+0000","last_clean":"2024-09-21T22:52:49.944888+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:49.944888+0000","last_undegraded":"2024-09-21T22:52:49.944888+0000","last_fullsized":"2024-09-21T22:52:49.944888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":293,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165166+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:50.165166+0000","last_peered":"2024-09-21T22:52:50.165166+0000","last_clean":"2024-09-21T22:52:50.165166+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:50.165166+0000","last_undegraded":"2024-09-21T22:52:50.165166+0000","last_fullsized":"2024-09-21T22:52:50.165166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.813092+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:49.813092+0000","last_peered":"2024-09-21T22:52:49.813092+0000","last_clean":"2024-09-21T22:52:49.813092+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:49.813092+0000","last_undegraded":"2024-09-21T22:52:49.813092+0000","last_fullsized":"2024-09-21T22:52:49.813092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3455,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164958+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:50.164958+0000","last_peered":"2024-09-21T22:52:50.164958+0000","last_clean":"2024-09-21T22:52:50.164958+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:50.164958+0000","last_undegraded":"2024-09-21T22:52:50.164958+0000","last_fullsized":"2024-09-21T22:52:50.164958+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165063+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:50.165063+0000","last_peered":"2024-09-21T22:52:50.165063+0000","last_clean":"2024-09-21T22:52:50.165063+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:50.165063+0000","last_undegraded":"2024-09-21T22:52:50.165063+0000","last_fullsized":"2024-09-21T22:52:50.165063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2808,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067203+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:48.067203+0000","last_peered":"2024-09-21T22:52:48.067203+0000","last_clean":"2024-09-21T22:52:48.067203+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:48.067203+0000","last_undegraded":"2024-09-21T22:52:48.067203+0000","last_fullsized":"2024-09-21T22:52:48.067203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":287,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066442+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:48.066442+0000","last_peered":"2024-09-21T22:52:48.066442+0000","last_clean":"2024-09-21T22:52:48.066442+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:48.066442+0000","last_undegraded":"2024-09-21T22:52:48.066442+0000","last_fullsized":"2024-09-21T22:52:48.066442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5573,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164899+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:50.164899+0000","last_peered":"2024-09-21T22:52:50.164899+0000","last_clean":"2024-09-21T22:52:50.164899+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:50.164899+0000","last_undegraded":"2024-09-21T22:52:50.164899+0000","last_fullsized":"2024-09-21T22:52:50.164899+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6402,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944777+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:49.944777+0000","last_peered":"2024-09-21T22:52:49.944777+0000","last_clean":"2024-09-21T22:52:49.944777+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:49.944777+0000","last_undegraded":"2024-09-21T22:52:49.944777+0000","last_fullsized":"2024-09-21T22:52:49.944777+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812495+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:49.812495+0000","last_peered":"2024-09-21T22:52:49.812495+0000","last_clean":"2024-09-21T22:52:49.812495+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:49.812495+0000","last_undegraded":"2024-09-21T22:52:49.812495+0000","last_fullsized":"2024-09-21T22:52:49.812495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066559+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:48.066559+0000","last_peered":"2024-09-21T22:52:48.066559+0000","last_clean":"2024-09-21T22:52:48.066559+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:48.066559+0000","last_undegraded":"2024-09-21T22:52:48.066559+0000","last_fullsized":"2024-09-21T22:52:48.066559+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812522+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:49.812522+0000","last_peered":"2024-09-21T22:52:49.812522+0000","last_clean":"2024-09-21T22:52:49.812522+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:49.812522+0000","last_undegraded":"2024-09-21T22:52:49.812522+0000","last_fullsized":"2024-09-21T22:52:49.812522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066073+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:48.066073+0000","last_peered":"2024-09-21T22:52:48.066073+0000","last_clean":"2024-09-21T22:52:48.066073+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:48.066073+0000","last_undegraded":"2024-09-21T22:52:48.066073+0000","last_fullsized":"2024-09-21T22:52:48.066073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2049,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066678+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:48.066678+0000","last_peered":"2024-09-21T22:52:48.066678+0000","last_clean":"2024-09-21T22:52:48.066678+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:48.066678+0000","last_undegraded":"2024-09-21T22:52:48.066678+0000","last_fullsized":"2024-09-21T22:52:48.066678+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067260+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:48.067260+0000","last_peered":"2024-09-21T22:52:48.067260+0000","last_clean":"2024-09-21T22:52:48.067260+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:48.067260+0000","last_undegraded":"2024-09-21T22:52:48.067260+0000","last_fullsized":"2024-09-21T22:52:48.067260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4070,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812817+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:49.812817+0000","last_peered":"2024-09-21T22:52:49.812817+0000","last_clean":"2024-09-21T22:52:49.812817+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:49.812817+0000","last_undegraded":"2024-09-21T22:52:49.812817+0000","last_fullsized":"2024-09-21T22:52:49.812817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7328,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812891+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:49.812891+0000","last_peered":"2024-09-21T22:52:49.812891+0000","last_clean":"2024-09-21T22:52:49.812891+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:49.812891+0000","last_undegraded":"2024-09-21T22:52:49.812891+0000","last_fullsized":"2024-09-21T22:52:49.812891+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5957,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812849+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:49.812849+0000","last_peered":"2024-09-21T22:52:49.812849+0000","last_clean":"2024-09-21T22:52:49.812849+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:49.812849+0000","last_undegraded":"2024-09-21T22:52:49.812849+0000","last_fullsized":"2024-09-21T22:52:49.812849+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7564,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066385+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:48.066385+0000","last_peered":"2024-09-21T22:52:48.066385+0000","last_clean":"2024-09-21T22:52:48.066385+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:48.066385+0000","last_undegraded":"2024-09-21T22:52:48.066385+0000","last_fullsized":"2024-09-21T22:52:48.066385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6605,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812366+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:49.812366+0000","last_peered":"2024-09-21T22:52:49.812366+0000","last_clean":"2024-09-21T22:52:49.812366+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:49.812366+0000","last_undegraded":"2024-09-21T22:52:49.812366+0000","last_fullsized":"2024-09-21T22:52:49.812366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944766+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:49.944766+0000","last_peered":"2024-09-21T22:52:49.944766+0000","last_clean":"2024-09-21T22:52:49.944766+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:49.944766+0000","last_undegraded":"2024-09-21T22:52:49.944766+0000","last_fullsized":"2024-09-21T22:52:49.944766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8910,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812550+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:49.812550+0000","last_peered":"2024-09-21T22:52:49.812550+0000","last_clean":"2024-09-21T22:52:49.812550+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:49.812550+0000","last_undegraded":"2024-09-21T22:52:49.812550+0000","last_fullsized":"2024-09-21T22:52:49.812550+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1871872,"data_stored":892988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626461,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":1824,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55244,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316596,"statfs":{"total":96636764160,"available":96580194304,"internally_reserved":0,"allocated":1888256,"data_stored":898878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:51.465 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T22:52:51.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:51.739 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:51.740 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:51.756 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-21T22:52:51.319268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411708,"kb_used_data":15816,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563012,"statfs":{"total":773094113280,"available":772672524288,"internally_reserved":0,"allocated":16195584,"data_stored":8348048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.962042"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8155,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067136+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:48.067136+0000","last_peered":"2024-09-21T22:52:48.067136+0000","last_clean":"2024-09-21T22:52:48.067136+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:48.067136+0000","last_undegraded":"2024-09-21T22:52:48.067136+0000","last_fullsized":"2024-09-21T22:52:48.067136+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3056,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066965+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:48.066965+0000","last_peered":"2024-09-21T22:52:48.066965+0000","last_clean":"2024-09-21T22:52:48.066965+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:48.066965+0000","last_undegraded":"2024-09-21T22:52:48.066965+0000","last_fullsized":"2024-09-21T22:52:48.066965+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7371,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944885+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:49.944885+0000","last_peered":"2024-09-21T22:52:49.944885+0000","last_clean":"2024-09-21T22:52:49.944885+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:49.944885+0000","last_undegraded":"2024-09-21T22:52:49.944885+0000","last_fullsized":"2024-09-21T22:52:49.944885+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9392,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165149+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:50.165149+0000","last_peered":"2024-09-21T22:52:50.165149+0000","last_clean":"2024-09-21T22:52:50.165149+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:50.165149+0000","last_undegraded":"2024-09-21T22:52:50.165149+0000","last_fullsized":"2024-09-21T22:52:50.165149+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3513,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944896+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:49.944896+0000","last_peered":"2024-09-21T22:52:49.944896+0000","last_clean":"2024-09-21T22:52:49.944896+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:49.944896+0000","last_undegraded":"2024-09-21T22:52:49.944896+0000","last_fullsized":"2024-09-21T22:52:49.944896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8753,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067097+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:48.067097+0000","last_peered":"2024-09-21T22:52:48.067097+0000","last_clean":"2024-09-21T22:52:48.067097+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:48.067097+0000","last_undegraded":"2024-09-21T22:52:48.067097+0000","last_fullsized":"2024-09-21T22:52:48.067097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11524,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812813+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:49.812813+0000","last_peered":"2024-09-21T22:52:49.812813+0000","last_clean":"2024-09-21T22:52:49.812813+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:49.812813+0000","last_undegraded":"2024-09-21T22:52:49.812813+0000","last_fullsized":"2024-09-21T22:52:49.812813+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":252,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066972+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:48.066972+0000","last_peered":"2024-09-21T22:52:48.066972+0000","last_clean":"2024-09-21T22:52:48.066972+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:48.066972+0000","last_undegraded":"2024-09-21T22:52:48.066972+0000","last_fullsized":"2024-09-21T22:52:48.066972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6768,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944888+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:49.944888+0000","last_peered":"2024-09-21T22:52:49.944888+0000","last_clean":"2024-09-21T22:52:49.944888+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:49.944888+0000","last_undegraded":"2024-09-21T22:52:49.944888+0000","last_fullsized":"2024-09-21T22:52:49.944888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":293,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165166+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:50.165166+0000","last_peered":"2024-09-21T22:52:50.165166+0000","last_clean":"2024-09-21T22:52:50.165166+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:50.165166+0000","last_undegraded":"2024-09-21T22:52:50.165166+0000","last_fullsized":"2024-09-21T22:52:50.165166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.813092+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:49.813092+0000","last_peered":"2024-09-21T22:52:49.813092+0000","last_clean":"2024-09-21T22:52:49.813092+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:49.813092+0000","last_undegraded":"2024-09-21T22:52:49.813092+0000","last_fullsized":"2024-09-21T22:52:49.813092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3455,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164958+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:50.164958+0000","last_peered":"2024-09-21T22:52:50.164958+0000","last_clean":"2024-09-21T22:52:50.164958+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:50.164958+0000","last_undegraded":"2024-09-21T22:52:50.164958+0000","last_fullsized":"2024-09-21T22:52:50.164958+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165063+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:50.165063+0000","last_peered":"2024-09-21T22:52:50.165063+0000","last_clean":"2024-09-21T22:52:50.165063+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:50.165063+0000","last_undegraded":"2024-09-21T22:52:50.165063+0000","last_fullsized":"2024-09-21T22:52:50.165063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2808,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067203+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:48.067203+0000","last_peered":"2024-09-21T22:52:48.067203+0000","last_clean":"2024-09-21T22:52:48.067203+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:48.067203+0000","last_undegraded":"2024-09-21T22:52:48.067203+0000","last_fullsized":"2024-09-21T22:52:48.067203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":287,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066442+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:48.066442+0000","last_peered":"2024-09-21T22:52:48.066442+0000","last_clean":"2024-09-21T22:52:48.066442+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:48.066442+0000","last_undegraded":"2024-09-21T22:52:48.066442+0000","last_fullsized":"2024-09-21T22:52:48.066442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5573,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164899+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:50.164899+0000","last_peered":"2024-09-21T22:52:50.164899+0000","last_clean":"2024-09-21T22:52:50.164899+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:50.164899+0000","last_undegraded":"2024-09-21T22:52:50.164899+0000","last_fullsized":"2024-09-21T22:52:50.164899+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6402,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944777+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:49.944777+0000","last_peered":"2024-09-21T22:52:49.944777+0000","last_clean":"2024-09-21T22:52:49.944777+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:49.944777+0000","last_undegraded":"2024-09-21T22:52:49.944777+0000","last_fullsized":"2024-09-21T22:52:49.944777+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812495+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:49.812495+0000","last_peered":"2024-09-21T22:52:49.812495+0000","last_clean":"2024-09-21T22:52:49.812495+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:49.812495+0000","last_undegraded":"2024-09-21T22:52:49.812495+0000","last_fullsized":"2024-09-21T22:52:49.812495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066559+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:48.066559+0000","last_peered":"2024-09-21T22:52:48.066559+0000","last_clean":"2024-09-21T22:52:48.066559+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:48.066559+0000","last_undegraded":"2024-09-21T22:52:48.066559+0000","last_fullsized":"2024-09-21T22:52:48.066559+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812522+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:49.812522+0000","last_peered":"2024-09-21T22:52:49.812522+0000","last_clean":"2024-09-21T22:52:49.812522+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:49.812522+0000","last_undegraded":"2024-09-21T22:52:49.812522+0000","last_fullsized":"2024-09-21T22:52:49.812522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066073+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:48.066073+0000","last_peered":"2024-09-21T22:52:48.066073+0000","last_clean":"2024-09-21T22:52:48.066073+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:48.066073+0000","last_undegraded":"2024-09-21T22:52:48.066073+0000","last_fullsized":"2024-09-21T22:52:48.066073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2049,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066678+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:48.066678+0000","last_peered":"2024-09-21T22:52:48.066678+0000","last_clean":"2024-09-21T22:52:48.066678+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:48.066678+0000","last_undegraded":"2024-09-21T22:52:48.066678+0000","last_fullsized":"2024-09-21T22:52:48.066678+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.067260+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:48.067260+0000","last_peered":"2024-09-21T22:52:48.067260+0000","last_clean":"2024-09-21T22:52:48.067260+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:48.067260+0000","last_undegraded":"2024-09-21T22:52:48.067260+0000","last_fullsized":"2024-09-21T22:52:48.067260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4070,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812817+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:49.812817+0000","last_peered":"2024-09-21T22:52:49.812817+0000","last_clean":"2024-09-21T22:52:49.812817+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:49.812817+0000","last_undegraded":"2024-09-21T22:52:49.812817+0000","last_fullsized":"2024-09-21T22:52:49.812817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7328,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812891+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:49.812891+0000","last_peered":"2024-09-21T22:52:49.812891+0000","last_clean":"2024-09-21T22:52:49.812891+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:49.812891+0000","last_undegraded":"2024-09-21T22:52:49.812891+0000","last_fullsized":"2024-09-21T22:52:49.812891+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5957,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812849+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:49.812849+0000","last_peered":"2024-09-21T22:52:49.812849+0000","last_clean":"2024-09-21T22:52:49.812849+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:49.812849+0000","last_undegraded":"2024-09-21T22:52:49.812849+0000","last_fullsized":"2024-09-21T22:52:49.812849+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7564,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.066385+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:48.066385+0000","last_peered":"2024-09-21T22:52:48.066385+0000","last_clean":"2024-09-21T22:52:48.066385+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:48.066385+0000","last_undegraded":"2024-09-21T22:52:48.066385+0000","last_fullsized":"2024-09-21T22:52:48.066385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6605,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812366+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:49.812366+0000","last_peered":"2024-09-21T22:52:49.812366+0000","last_clean":"2024-09-21T22:52:49.812366+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:49.812366+0000","last_undegraded":"2024-09-21T22:52:49.812366+0000","last_fullsized":"2024-09-21T22:52:49.812366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944766+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:49.944766+0000","last_peered":"2024-09-21T22:52:49.944766+0000","last_clean":"2024-09-21T22:52:49.944766+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:49.944766+0000","last_undegraded":"2024-09-21T22:52:49.944766+0000","last_fullsized":"2024-09-21T22:52:49.944766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8910,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812550+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:49.812550+0000","last_peered":"2024-09-21T22:52:49.812550+0000","last_clean":"2024-09-21T22:52:49.812550+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:49.812550+0000","last_undegraded":"2024-09-21T22:52:49.812550+0000","last_fullsized":"2024-09-21T22:52:49.812550+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1871872,"data_stored":892988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626461,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":1824,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":1867776,"data_stored":892969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55244,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316596,"statfs":{"total":96636764160,"available":96580194304,"internally_reserved":0,"allocated":1888256,"data_stored":898878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:51.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T22:52:52.036 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:52.036 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":157,"fsid":"56013142-0739-4aee-ba29-30eddcc2dbe0","created":"2024-09-21T22:49:36.309693+0000","modified":"2024-09-21T22:52:48.381852+0000","last_up_change":"2024-09-21T22:51:41.430550+0000","last_in_change":"2024-09-21T22:52:32.286494+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T22:49:51.363777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T22:49:55.226638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T22:50:05.084190+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ecfcea04-7198-4c57-bc74-45aaf791e3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6809","nonce":2211646084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6811","nonce":2211646084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6815","nonce":2211646084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2211646084},{"type":"v1","addr":"172.21.15.94:6813","nonce":2211646084}]},"public_addr":"172.21.15.94:6809/2211646084","cluster_addr":"172.21.15.94:6811/2211646084","heartbeat_back_addr":"172.21.15.94:6815/2211646084","heartbeat_front_addr":"172.21.15.94:6813/2211646084","state":["exists","up"]},{"osd":1,"uuid":"b6b749e6-4a66-4d8b-bf50-c9634a6927d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6825","nonce":1102146551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6827","nonce":1102146551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6831","nonce":1102146551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1102146551},{"type":"v1","addr":"172.21.15.94:6829","nonce":1102146551}]},"public_addr":"172.21.15.94:6825/1102146551","cluster_addr":"172.21.15.94:6827/1102146551","heartbeat_back_addr":"172.21.15.94:6831/1102146551","heartbeat_front_addr":"172.21.15.94:6829/1102146551","state":["exists","up"]},{"osd":2,"uuid":"4e90da65-2fdd-4c85-a207-70d5f22c95a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6817","nonce":2482161893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6819","nonce":2482161893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6823","nonce":2482161893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2482161893},{"type":"v1","addr":"172.21.15.94:6821","nonce":2482161893}]},"public_addr":"172.21.15.94:6817/2482161893","cluster_addr":"172.21.15.94:6819/2482161893","heartbeat_back_addr":"172.21.15.94:6823/2482161893","heartbeat_front_addr":"172.21.15.94:6821/2482161893","state":["exists","up"]},{"osd":3,"uuid":"3c97651a-402c-4171-9648-c615dd754c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":77,"up_from":90,"up_thru":134,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6801","nonce":2575188997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6803","nonce":2575188997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6807","nonce":2575188997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2575188997},{"type":"v1","addr":"172.21.15.94:6805","nonce":2575188997}]},"public_addr":"172.21.15.94:6801/2575188997","cluster_addr":"172.21.15.94:6803/2575188997","heartbeat_back_addr":"172.21.15.94:6807/2575188997","heartbeat_front_addr":"172.21.15.94:6805/2575188997","state":["exists","up"]},{"osd":4,"uuid":"b935b00e-0d91-4f2b-9f2d-2c2be1144ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6821","nonce":1105568802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6823","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6826","nonce":1105568802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6833","nonce":1105568802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1105568802},{"type":"v1","addr":"172.21.15.99:6830","nonce":1105568802}]},"public_addr":"172.21.15.99:6821/1105568802","cluster_addr":"172.21.15.99:6826/1105568802","heartbeat_back_addr":"172.21.15.99:6833/1105568802","heartbeat_front_addr":"172.21.15.99:6830/1105568802","state":["exists","up"]},{"osd":5,"uuid":"559dea61-5abd-423e-9dfb-cf92f9e6d4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6820","nonce":2691947692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6824","nonce":2691947692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6829","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6831","nonce":2691947692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":2691947692},{"type":"v1","addr":"172.21.15.99:6827","nonce":2691947692}]},"public_addr":"172.21.15.99:6820/2691947692","cluster_addr":"172.21.15.99:6824/2691947692","heartbeat_back_addr":"172.21.15.99:6831/2691947692","heartbeat_front_addr":"172.21.15.99:6827/2691947692","state":["exists","up"]},{"osd":6,"uuid":"aeee922d-53c2-41f4-9433-8a9691f3e674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":42,"up_thru":147,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6811","nonce":3470582226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6813","nonce":3470582226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6817","nonce":3470582226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3470582226},{"type":"v1","addr":"172.21.15.99:6815","nonce":3470582226}]},"public_addr":"172.21.15.99:6811/3470582226","cluster_addr":"172.21.15.99:6813/3470582226","heartbeat_back_addr":"172.21.15.99:6817/3470582226","heartbeat_front_addr":"172.21.15.99:6815/3470582226","state":["exists","up"]},{"osd":7,"uuid":"b40a3234-6950-4a73-9a5c-f0c086450816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6801","nonce":1713641021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6803","nonce":1713641021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6807","nonce":1713641021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1713641021},{"type":"v1","addr":"172.21.15.99:6805","nonce":1713641021}]},"public_addr":"172.21.15.99:6801/1713641021","cluster_addr":"172.21.15.99:6803/1713641021","heartbeat_back_addr":"172.21.15.99:6807/1713641021","heartbeat_front_addr":"172.21.15.99:6805/1713641021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.846676+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:47.117051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.836357+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.351920+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.975489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.838876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.842751+0000","dead_epoch":24},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T22:49:46.784805+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1f","osds":[3,5]}],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T22:52:53.054 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T22:52:53.054 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:53.209 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:53.209 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:53.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T22:52:53.498 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T22:52:53.515 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T22:52:53.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:53.665 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:53.665 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:53.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T22:52:53.962 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T22:52:53.980 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T22:52:53.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:54.124 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:54.124 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:54.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T22:52:54.432 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T22:52:54.452 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T22:52:54.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:54.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T22:52:54.902 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T22:52:54.921 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T22:52:54.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:55.056 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:55.057 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:55.057 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:55.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T22:52:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T22:52:55.366 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T22:52:55.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:55.517 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:55.517 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:55.517 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:55.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T22:52:55.820 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T22:52:55.839 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T22:52:55.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:55.991 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:55.991 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:55.991 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:56.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T22:52:56.294 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T22:52:56.313 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T22:52:56.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T22:52:56.464 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-21T22:52:56.486 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T22:52:56.489 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-21T22:52:56.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T22:52:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T22:52:56.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:52:57.106 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:52:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:52:57.124 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-21T22:52:55.320521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21646,"ondisk_log_size":21646,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411720,"kb_used_data":15840,"kb_used_omap":55,"kb_used_meta":395656,"kb_avail":754563000,"statfs":{"total":773094113280,"available":772672512000,"internally_reserved":0,"allocated":16220160,"data_stored":8359790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":405152106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002608"},"pg_stats":[{"pgid":"3.1f","version":"137'1058","reported_seq":7459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388692+0000","last_change":"2024-09-21T22:52:28.268047+0000","last_active":"2024-09-21T22:52:48.388692+0000","last_peered":"2024-09-21T22:52:48.388692+0000","last_clean":"2024-09-21T22:52:48.388692+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:48.388692+0000","last_undegraded":"2024-09-21T22:52:48.388692+0000","last_fullsized":"2024-09-21T22:52:48.388692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:14.981735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"137'625","reported_seq":6543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388650+0000","last_change":"2024-09-21T22:52:28.270592+0000","last_active":"2024-09-21T22:52:48.388650+0000","last_peered":"2024-09-21T22:52:48.388650+0000","last_clean":"2024-09-21T22:52:48.388650+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:48.388650+0000","last_undegraded":"2024-09-21T22:52:48.388650+0000","last_fullsized":"2024-09-21T22:52:48.388650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:49.475440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"137'1101","reported_seq":8157,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545721+0000","last_change":"2024-09-21T22:52:40.092303+0000","last_active":"2024-09-21T22:52:49.545721+0000","last_peered":"2024-09-21T22:52:49.545721+0000","last_clean":"2024-09-21T22:52:49.545721+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:52:49.545721+0000","last_undegraded":"2024-09-21T22:52:49.545721+0000","last_fullsized":"2024-09-21T22:52:49.545721+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:25.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"137'852","reported_seq":3058,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545573+0000","last_change":"2024-09-21T22:52:37.070441+0000","last_active":"2024-09-21T22:52:49.545573+0000","last_peered":"2024-09-21T22:52:49.545573+0000","last_clean":"2024-09-21T22:52:49.545573+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:52:49.545573+0000","last_undegraded":"2024-09-21T22:52:49.545573+0000","last_fullsized":"2024-09-21T22:52:49.545573+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:55.471580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"134'1105","reported_seq":7371,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944885+0000","last_change":"2024-09-21T22:52:42.062658+0000","last_active":"2024-09-21T22:52:49.944885+0000","last_peered":"2024-09-21T22:52:49.944885+0000","last_clean":"2024-09-21T22:52:49.944885+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:49.944885+0000","last_undegraded":"2024-09-21T22:52:49.944885+0000","last_fullsized":"2024-09-21T22:52:49.944885+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:17.039295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"139'819","reported_seq":9400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388743+0000","last_change":"2024-09-21T22:52:36.072883+0000","last_active":"2024-09-21T22:52:48.388743+0000","last_peered":"2024-09-21T22:52:48.388743+0000","last_clean":"2024-09-21T22:52:48.388743+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:48.388743+0000","last_undegraded":"2024-09-21T22:52:48.388743+0000","last_fullsized":"2024-09-21T22:52:48.388743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:20.554553+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'561","reported_seq":9392,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165149+0000","last_change":"2024-09-21T22:52:28.280771+0000","last_active":"2024-09-21T22:52:50.165149+0000","last_peered":"2024-09-21T22:52:50.165149+0000","last_clean":"2024-09-21T22:52:50.165149+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:50.165149+0000","last_undegraded":"2024-09-21T22:52:50.165149+0000","last_fullsized":"2024-09-21T22:52:50.165149+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:26.572869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'1116","reported_seq":11500,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388248+0000","last_change":"2024-09-21T22:52:28.267633+0000","last_active":"2024-09-21T22:52:48.388248+0000","last_peered":"2024-09-21T22:52:48.388248+0000","last_clean":"2024-09-21T22:52:48.388248+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:48.388248+0000","last_undegraded":"2024-09-21T22:52:48.388248+0000","last_fullsized":"2024-09-21T22:52:48.388248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:37.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'507","reported_seq":3513,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944896+0000","last_change":"2024-09-21T22:52:28.269452+0000","last_active":"2024-09-21T22:52:49.944896+0000","last_peered":"2024-09-21T22:52:49.944896+0000","last_clean":"2024-09-21T22:52:49.944896+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:49.944896+0000","last_undegraded":"2024-09-21T22:52:49.944896+0000","last_fullsized":"2024-09-21T22:52:49.944896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:20.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"137'442","reported_seq":8755,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545803+0000","last_change":"2024-09-21T22:52:28.292436+0000","last_active":"2024-09-21T22:52:49.545803+0000","last_peered":"2024-09-21T22:52:49.545803+0000","last_clean":"2024-09-21T22:52:49.545803+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:49.545803+0000","last_undegraded":"2024-09-21T22:52:49.545803+0000","last_fullsized":"2024-09-21T22:52:49.545803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:41.990467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"134'494","reported_seq":11524,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812813+0000","last_change":"2024-09-21T22:52:28.282471+0000","last_active":"2024-09-21T22:52:49.812813+0000","last_peered":"2024-09-21T22:52:49.812813+0000","last_clean":"2024-09-21T22:52:49.812813+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:52:49.812813+0000","last_undegraded":"2024-09-21T22:52:49.812813+0000","last_fullsized":"2024-09-21T22:52:49.812813+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.494896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"137'497","reported_seq":9788,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389123+0000","last_change":"2024-09-21T22:52:36.073104+0000","last_active":"2024-09-21T22:52:48.389123+0000","last_peered":"2024-09-21T22:52:48.389123+0000","last_clean":"2024-09-21T22:52:48.389123+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:48.389123+0000","last_undegraded":"2024-09-21T22:52:48.389123+0000","last_fullsized":"2024-09-21T22:52:48.389123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":254,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.992453+0000","last_change":"2024-09-21T22:50:34.212199+0000","last_active":"2024-09-21T22:52:49.992453+0000","last_peered":"2024-09-21T22:52:49.992453+0000","last_clean":"2024-09-21T22:52:49.992453+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:49.992453+0000","last_undegraded":"2024-09-21T22:52:49.992453+0000","last_fullsized":"2024-09-21T22:52:49.992453+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:59.951331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"137'598","reported_seq":6768,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944888+0000","last_change":"2024-09-21T22:52:28.268165+0000","last_active":"2024-09-21T22:52:49.944888+0000","last_peered":"2024-09-21T22:52:49.944888+0000","last_clean":"2024-09-21T22:52:49.944888+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:49.944888+0000","last_undegraded":"2024-09-21T22:52:49.944888+0000","last_fullsized":"2024-09-21T22:52:49.944888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:28:22.783688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":293,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165166+0000","last_change":"2024-09-21T22:50:34.195127+0000","last_active":"2024-09-21T22:52:50.165166+0000","last_peered":"2024-09-21T22:52:50.165166+0000","last_clean":"2024-09-21T22:52:50.165166+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:50.165166+0000","last_undegraded":"2024-09-21T22:52:50.165166+0000","last_fullsized":"2024-09-21T22:52:50.165166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:53:38.537842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'531","reported_seq":4545,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388713+0000","last_change":"2024-09-21T22:52:28.271052+0000","last_active":"2024-09-21T22:52:48.388713+0000","last_peered":"2024-09-21T22:52:48.388713+0000","last_clean":"2024-09-21T22:52:48.388713+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:48.388713+0000","last_undegraded":"2024-09-21T22:52:48.388713+0000","last_fullsized":"2024-09-21T22:52:48.388713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:08.549635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.813092+0000","last_change":"2024-09-21T22:49:58.986824+0000","last_active":"2024-09-21T22:52:49.813092+0000","last_peered":"2024-09-21T22:52:49.813092+0000","last_clean":"2024-09-21T22:52:49.813092+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:49.813092+0000","last_undegraded":"2024-09-21T22:52:49.813092+0000","last_fullsized":"2024-09-21T22:52:49.813092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'628","reported_seq":3455,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164958+0000","last_change":"2024-09-21T22:52:35.328357+0000","last_active":"2024-09-21T22:52:50.164958+0000","last_peered":"2024-09-21T22:52:50.164958+0000","last_clean":"2024-09-21T22:52:50.164958+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:50.164958+0000","last_undegraded":"2024-09-21T22:52:50.164958+0000","last_fullsized":"2024-09-21T22:52:50.164958+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:12.682816+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.165063+0000","last_change":"2024-09-21T22:49:58.995224+0000","last_active":"2024-09-21T22:52:50.165063+0000","last_peered":"2024-09-21T22:52:50.165063+0000","last_clean":"2024-09-21T22:52:50.165063+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:50.165063+0000","last_undegraded":"2024-09-21T22:52:50.165063+0000","last_fullsized":"2024-09-21T22:52:50.165063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:15.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"134'579","reported_seq":2810,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545371+0000","last_change":"2024-09-21T22:52:28.291417+0000","last_active":"2024-09-21T22:52:49.545371+0000","last_peered":"2024-09-21T22:52:49.545371+0000","last_clean":"2024-09-21T22:52:49.545371+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:49.545371+0000","last_undegraded":"2024-09-21T22:52:49.545371+0000","last_fullsized":"2024-09-21T22:52:49.545371+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:11.715558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'733","reported_seq":7109,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389061+0000","last_change":"2024-09-21T22:52:38.073867+0000","last_active":"2024-09-21T22:52:48.389061+0000","last_peered":"2024-09-21T22:52:48.389061+0000","last_clean":"2024-09-21T22:52:48.389061+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:48.389061+0000","last_undegraded":"2024-09-21T22:52:48.389061+0000","last_fullsized":"2024-09-21T22:52:48.389061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:09:38.135756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545758+0000","last_change":"2024-09-21T22:49:58.992797+0000","last_active":"2024-09-21T22:52:49.545758+0000","last_peered":"2024-09-21T22:52:49.545758+0000","last_clean":"2024-09-21T22:52:49.545758+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:49.545758+0000","last_undegraded":"2024-09-21T22:52:49.545758+0000","last_fullsized":"2024-09-21T22:52:49.545758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:18.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"139'695","reported_seq":5573,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:50.164899+0000","last_change":"2024-09-21T22:52:35.329600+0000","last_active":"2024-09-21T22:52:50.164899+0000","last_peered":"2024-09-21T22:52:50.164899+0000","last_clean":"2024-09-21T22:52:50.164899+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:50.164899+0000","last_undegraded":"2024-09-21T22:52:50.164899+0000","last_fullsized":"2024-09-21T22:52:50.164899+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:32.179554+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.389211+0000","last_change":"2024-09-21T22:52:32.305054+0000","last_active":"2024-09-21T22:52:48.389211+0000","last_peered":"2024-09-21T22:52:48.389211+0000","last_clean":"2024-09-21T22:52:48.389211+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:48.389211+0000","last_undegraded":"2024-09-21T22:52:48.389211+0000","last_fullsized":"2024-09-21T22:52:48.389211+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:51.687404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'411","reported_seq":6402,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944777+0000","last_change":"2024-09-21T22:52:40.088553+0000","last_active":"2024-09-21T22:52:49.944777+0000","last_peered":"2024-09-21T22:52:49.944777+0000","last_clean":"2024-09-21T22:52:49.944777+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:49.944777+0000","last_undegraded":"2024-09-21T22:52:49.944777+0000","last_fullsized":"2024-09-21T22:52:49.944777+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:16:18.422029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812495+0000","last_change":"2024-09-21T22:49:58.986710+0000","last_active":"2024-09-21T22:52:49.812495+0000","last_peered":"2024-09-21T22:52:49.812495+0000","last_clean":"2024-09-21T22:52:49.812495+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:49.812495+0000","last_undegraded":"2024-09-21T22:52:49.812495+0000","last_fullsized":"2024-09-21T22:52:49.812495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:06.288614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"136'723","reported_seq":3249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388253+0000","last_change":"2024-09-21T22:52:28.269693+0000","last_active":"2024-09-21T22:52:48.388253+0000","last_peered":"2024-09-21T22:52:48.388253+0000","last_clean":"2024-09-21T22:52:48.388253+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:52:48.388253+0000","last_undegraded":"2024-09-21T22:52:48.388253+0000","last_fullsized":"2024-09-21T22:52:48.388253+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:17.503219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":306,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545550+0000","last_change":"2024-09-21T22:52:32.321595+0000","last_active":"2024-09-21T22:52:49.545550+0000","last_peered":"2024-09-21T22:52:49.545550+0000","last_clean":"2024-09-21T22:52:49.545550+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:52:49.545550+0000","last_undegraded":"2024-09-21T22:52:49.545550+0000","last_fullsized":"2024-09-21T22:52:49.545550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:55.937233+0000","last_clean_scrub_stamp":"2024-09-21T22:49:55.937233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:25.872949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812522+0000","last_change":"2024-09-21T22:49:53.905125+0000","last_active":"2024-09-21T22:52:49.812522+0000","last_peered":"2024-09-21T22:52:49.812522+0000","last_clean":"2024-09-21T22:52:49.812522+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:49.812522+0000","last_undegraded":"2024-09-21T22:52:49.812522+0000","last_fullsized":"2024-09-21T22:52:49.812522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:49:51.908308+0000","last_clean_scrub_stamp":"2024-09-21T22:49:51.908308+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:27.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'803","reported_seq":3155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.992484+0000","last_change":"2024-09-21T22:52:28.294026+0000","last_active":"2024-09-21T22:52:49.992484+0000","last_peered":"2024-09-21T22:52:49.992484+0000","last_clean":"2024-09-21T22:52:49.992484+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:49.992484+0000","last_undegraded":"2024-09-21T22:52:49.992484+0000","last_fullsized":"2024-09-21T22:52:49.992484+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:52.682541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"138'836","reported_seq":6349,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388660+0000","last_change":"2024-09-21T22:52:38.074743+0000","last_active":"2024-09-21T22:52:48.388660+0000","last_peered":"2024-09-21T22:52:48.388660+0000","last_clean":"2024-09-21T22:52:48.388660+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:48.388660+0000","last_undegraded":"2024-09-21T22:52:48.388660+0000","last_fullsized":"2024-09-21T22:52:48.388660+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:03.443006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"136'363","reported_seq":2051,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.992420+0000","last_change":"2024-09-21T22:52:39.088498+0000","last_active":"2024-09-21T22:52:49.992420+0000","last_peered":"2024-09-21T22:52:49.992420+0000","last_clean":"2024-09-21T22:52:49.992420+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:52:49.992420+0000","last_undegraded":"2024-09-21T22:52:49.992420+0000","last_fullsized":"2024-09-21T22:52:49.992420+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:44:21.082349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"131'385","reported_seq":1729,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:48.388380+0000","last_change":"2024-09-21T22:52:28.269527+0000","last_active":"2024-09-21T22:52:48.388380+0000","last_peered":"2024-09-21T22:52:48.388380+0000","last_clean":"2024-09-21T22:52:48.388380+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:48.388380+0000","last_undegraded":"2024-09-21T22:52:48.388380+0000","last_fullsized":"2024-09-21T22:52:48.388380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:55:57.987027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"137'354","reported_seq":5493,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.545629+0000","last_change":"2024-09-21T22:52:28.294973+0000","last_active":"2024-09-21T22:52:49.545629+0000","last_peered":"2024-09-21T22:52:49.545629+0000","last_clean":"2024-09-21T22:52:49.545629+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:49.545629+0000","last_undegraded":"2024-09-21T22:52:49.545629+0000","last_fullsized":"2024-09-21T22:52:49.545629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:30.046401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'715","reported_seq":4070,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812817+0000","last_change":"2024-09-21T22:52:28.283712+0000","last_active":"2024-09-21T22:52:49.812817+0000","last_peered":"2024-09-21T22:52:49.812817+0000","last_clean":"2024-09-21T22:52:49.812817+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:49.812817+0000","last_undegraded":"2024-09-21T22:52:49.812817+0000","last_fullsized":"2024-09-21T22:52:49.812817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:05.166648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"137'689","reported_seq":7328,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812891+0000","last_change":"2024-09-21T22:52:39.086519+0000","last_active":"2024-09-21T22:52:49.812891+0000","last_peered":"2024-09-21T22:52:49.812891+0000","last_clean":"2024-09-21T22:52:49.812891+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:52:49.812891+0000","last_undegraded":"2024-09-21T22:52:49.812891+0000","last_fullsized":"2024-09-21T22:52:49.812891+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:19:34.015406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"137'484","reported_seq":5957,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812849+0000","last_change":"2024-09-21T22:52:37.069411+0000","last_active":"2024-09-21T22:52:49.812849+0000","last_peered":"2024-09-21T22:52:49.812849+0000","last_clean":"2024-09-21T22:52:49.812849+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:52:49.812849+0000","last_undegraded":"2024-09-21T22:52:49.812849+0000","last_fullsized":"2024-09-21T22:52:49.812849+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:18.676867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"136'643","reported_seq":7566,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.992470+0000","last_change":"2024-09-21T22:52:28.295622+0000","last_active":"2024-09-21T22:52:49.992470+0000","last_peered":"2024-09-21T22:52:49.992470+0000","last_clean":"2024-09-21T22:52:49.992470+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:49.992470+0000","last_undegraded":"2024-09-21T22:52:49.992470+0000","last_fullsized":"2024-09-21T22:52:49.992470+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.849861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'968","reported_seq":6605,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812366+0000","last_change":"2024-09-21T22:52:41.068413+0000","last_active":"2024-09-21T22:52:49.812366+0000","last_peered":"2024-09-21T22:52:49.812366+0000","last_clean":"2024-09-21T22:52:49.812366+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:52:49.812366+0000","last_undegraded":"2024-09-21T22:52:49.812366+0000","last_fullsized":"2024-09-21T22:52:49.812366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:23.830499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'600","reported_seq":8651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.944766+0000","last_change":"2024-09-21T22:52:28.267323+0000","last_active":"2024-09-21T22:52:49.944766+0000","last_peered":"2024-09-21T22:52:49.944766+0000","last_clean":"2024-09-21T22:52:49.944766+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:49.944766+0000","last_undegraded":"2024-09-21T22:52:49.944766+0000","last_fullsized":"2024-09-21T22:52:49.944766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:40.572444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"139'611","reported_seq":8910,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:49.812550+0000","last_change":"2024-09-21T22:52:35.327529+0000","last_active":"2024-09-21T22:52:49.812550+0000","last_peered":"2024-09-21T22:52:49.812550+0000","last_clean":"2024-09-21T22:52:49.812550+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:52:49.812550+0000","last_undegraded":"2024-09-21T22:52:49.812550+0000","last_fullsized":"2024-09-21T22:52:49.812550+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:50:06.007898+0000","last_clean_scrub_stamp":"2024-09-21T22:50:06.007898+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:27.692469+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476776,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55804,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626462,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42432,"kb_used_data":1836,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329408,"statfs":{"total":96636764160,"available":96593313792,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83012,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288828,"statfs":{"total":96636764160,"available":96551759872,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55244,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316596,"statfs":{"total":96636764160,"available":96580194304,"internally_reserved":0,"allocated":1888256,"data_stored":898878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":2461696,"data_stored":1483337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39488,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332352,"statfs":{"total":96636764160,"available":96596328448,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:52:57.128 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.128 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.128 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.128 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.128 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.129 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.130 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T22:49:55.937233+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=55, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T22:49:51.908308+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=51, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.131 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T22:50:06.007898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=50, tm_sec=6, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T22:52:57.132 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T22:53:17.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T22:53:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-21T22:53:17.442 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-21T22:53:17.458 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-21T22:53:17.327404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":553,"num_read":220916,"num_read_kb":717086,"num_write":24472,"num_write_kb":334330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5400,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22206,"ondisk_log_size":22206,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":415864,"kb_used_data":15864,"kb_used_omap":55,"kb_used_meta":399816,"kb_avail":754558856,"statfs":{"total":773094113280,"available":772668268544,"internally_reserved":0,"allocated":16244736,"data_stored":8371532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56982,"internal_metadata":409411946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003738"},"pg_stats":[{"pgid":"3.1f","version":"157'1072","reported_seq":7482,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.830593+0000","last_change":"2024-09-21T22:52:58.830696+0000","last_active":"2024-09-21T22:52:58.830593+0000","last_peered":"2024-09-21T22:52:58.830593+0000","last_clean":"2024-09-21T22:52:58.830593+0000","last_became_active":"2024-09-21T22:51:42.457626+0000","last_became_peered":"2024-09-21T22:51:42.457626+0000","last_unstale":"2024-09-21T22:52:58.830593+0000","last_undegraded":"2024-09-21T22:52:58.830593+0000","last_fullsized":"2024-09-21T22:52:58.830593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'1072","last_scrub_stamp":"2024-09-21T22:52:58.830365+0000","last_deep_scrub":"157'1072","last_deep_scrub_stamp":"2024-09-21T22:52:58.830365+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.830365+0000","objects_scrubbed":14,"log_size":1072,"log_dups_size":0,"ondisk_log_size":1072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:19:53.822815+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0022041470000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9922,"num_read_kb":22030,"num_write":1297,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4149425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"157'651","reported_seq":6579,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:59.824884+0000","last_change":"2024-09-21T22:52:59.824948+0000","last_active":"2024-09-21T22:52:59.824884+0000","last_peered":"2024-09-21T22:52:59.824884+0000","last_clean":"2024-09-21T22:52:59.824884+0000","last_became_active":"2024-09-21T22:51:42.459072+0000","last_became_peered":"2024-09-21T22:51:42.459072+0000","last_unstale":"2024-09-21T22:52:59.824884+0000","last_undegraded":"2024-09-21T22:52:59.824884+0000","last_fullsized":"2024-09-21T22:52:59.824884+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'651","last_scrub_stamp":"2024-09-21T22:52:59.824680+0000","last_deep_scrub":"157'651","last_deep_scrub_stamp":"2024-09-21T22:52:59.824680+0000","last_clean_scrub_stamp":"2024-09-21T22:52:59.824680+0000","objects_scrubbed":26,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:01.701609+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0046288090000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7862,"num_read_kb":20917,"num_write":536,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":92316,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"157'1124","reported_seq":8189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:02.594724+0000","last_change":"2024-09-21T22:53:02.594836+0000","last_active":"2024-09-21T22:53:02.594724+0000","last_peered":"2024-09-21T22:53:02.594724+0000","last_clean":"2024-09-21T22:53:02.594724+0000","last_became_active":"2024-09-21T22:52:40.089654+0000","last_became_peered":"2024-09-21T22:52:40.089654+0000","last_unstale":"2024-09-21T22:53:02.594724+0000","last_undegraded":"2024-09-21T22:53:02.594724+0000","last_fullsized":"2024-09-21T22:53:02.594724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'1124","last_scrub_stamp":"2024-09-21T22:53:02.594516+0000","last_deep_scrub":"157'1124","last_deep_scrub_stamp":"2024-09-21T22:53:02.594516+0000","last_clean_scrub_stamp":"2024-09-21T22:53:02.594516+0000","objects_scrubbed":23,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:50:36.347309+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0038761099999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10762,"num_read_kb":47510,"num_write":1539,"num_write_kb":18458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":66581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"157'881","reported_seq":3097,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:01.605219+0000","last_change":"2024-09-21T22:53:01.605298+0000","last_active":"2024-09-21T22:53:01.605219+0000","last_peered":"2024-09-21T22:53:01.605219+0000","last_clean":"2024-09-21T22:53:01.605219+0000","last_became_active":"2024-09-21T22:52:37.067703+0000","last_became_peered":"2024-09-21T22:52:37.067703+0000","last_unstale":"2024-09-21T22:53:01.605219+0000","last_undegraded":"2024-09-21T22:53:01.605219+0000","last_fullsized":"2024-09-21T22:53:01.605219+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'881","last_scrub_stamp":"2024-09-21T22:53:01.605015+0000","last_deep_scrub":"157'881","last_deep_scrub_stamp":"2024-09-21T22:53:01.605015+0000","last_clean_scrub_stamp":"2024-09-21T22:53:01.605015+0000","objects_scrubbed":29,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:33:12.329726+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.0020365740000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":3859,"num_read_kb":11189,"num_write":1126,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16384,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"157'1117","reported_seq":7392,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.941493+0000","last_change":"2024-09-21T22:52:57.941663+0000","last_active":"2024-09-21T22:52:57.941493+0000","last_peered":"2024-09-21T22:52:57.941493+0000","last_clean":"2024-09-21T22:52:57.941493+0000","last_became_active":"2024-09-21T22:52:42.061172+0000","last_became_peered":"2024-09-21T22:52:42.061172+0000","last_unstale":"2024-09-21T22:52:57.941493+0000","last_undegraded":"2024-09-21T22:52:57.941493+0000","last_fullsized":"2024-09-21T22:52:57.941493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'1117","last_scrub_stamp":"2024-09-21T22:52:57.941203+0000","last_deep_scrub":"157'1117","last_deep_scrub_stamp":"2024-09-21T22:52:57.941203+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.941203+0000","objects_scrubbed":12,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:21:57.349148+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.002804969,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9623,"num_read_kb":37856,"num_write":1619,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":63580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'831","reported_seq":9421,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:55.995694+0000","last_change":"2024-09-21T22:52:55.995776+0000","last_active":"2024-09-21T22:52:55.995694+0000","last_peered":"2024-09-21T22:52:55.995694+0000","last_clean":"2024-09-21T22:52:55.995694+0000","last_became_active":"2024-09-21T22:52:36.069027+0000","last_became_peered":"2024-09-21T22:52:36.069027+0000","last_unstale":"2024-09-21T22:52:55.995694+0000","last_undegraded":"2024-09-21T22:52:55.995694+0000","last_fullsized":"2024-09-21T22:52:55.995694+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'831","last_scrub_stamp":"2024-09-21T22:52:55.995501+0000","last_deep_scrub":"157'831","last_deep_scrub_stamp":"2024-09-21T22:52:55.995501+0000","last_clean_scrub_stamp":"2024-09-21T22:52:55.995501+0000","objects_scrubbed":12,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:23.152246+0000","scrub_duration":23,"objects_trimmed":4,"snaptrim_duration":0.0058365680000000003,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":10046,"num_read_kb":13202,"num_write":1121,"num_write_kb":4814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":101129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"157'579","reported_seq":9419,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:56.416919+0000","last_change":"2024-09-21T22:52:56.416973+0000","last_active":"2024-09-21T22:52:56.416919+0000","last_peered":"2024-09-21T22:52:56.416919+0000","last_clean":"2024-09-21T22:52:56.416919+0000","last_became_active":"2024-09-21T22:52:23.210200+0000","last_became_peered":"2024-09-21T22:52:23.210200+0000","last_unstale":"2024-09-21T22:52:56.416919+0000","last_undegraded":"2024-09-21T22:52:56.416919+0000","last_fullsized":"2024-09-21T22:52:56.416919+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'579","last_scrub_stamp":"2024-09-21T22:52:56.416836+0000","last_deep_scrub":"157'579","last_deep_scrub_stamp":"2024-09-21T22:52:56.416836+0000","last_clean_scrub_stamp":"2024-09-21T22:52:56.416836+0000","objects_scrubbed":18,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:04.178381+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.00091456600000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8902,"num_read_kb":61109,"num_write":536,"num_write_kb":24718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":22473252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"157'1137","reported_seq":11530,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:56.805008+0000","last_change":"2024-09-21T22:52:56.805069+0000","last_active":"2024-09-21T22:52:56.805008+0000","last_peered":"2024-09-21T22:52:56.805008+0000","last_clean":"2024-09-21T22:52:56.805008+0000","last_became_active":"2024-09-21T22:51:42.452614+0000","last_became_peered":"2024-09-21T22:51:42.452614+0000","last_unstale":"2024-09-21T22:52:56.805008+0000","last_undegraded":"2024-09-21T22:52:56.805008+0000","last_fullsized":"2024-09-21T22:52:56.805008+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'1137","last_scrub_stamp":"2024-09-21T22:52:56.804929+0000","last_deep_scrub":"157'1137","last_deep_scrub_stamp":"2024-09-21T22:52:56.804929+0000","last_clean_scrub_stamp":"2024-09-21T22:52:56.804929+0000","objects_scrubbed":21,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:49.881597+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0019099259999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":14388,"num_read_kb":27721,"num_write":1685,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"157'524","reported_seq":3539,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:55.869972+0000","last_change":"2024-09-21T22:52:55.870028+0000","last_active":"2024-09-21T22:52:55.869972+0000","last_peered":"2024-09-21T22:52:55.869972+0000","last_clean":"2024-09-21T22:52:55.869972+0000","last_became_active":"2024-09-21T22:50:07.037516+0000","last_became_peered":"2024-09-21T22:50:07.037516+0000","last_unstale":"2024-09-21T22:52:55.869972+0000","last_undegraded":"2024-09-21T22:52:55.869972+0000","last_fullsized":"2024-09-21T22:52:55.869972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'524","last_scrub_stamp":"2024-09-21T22:52:55.869886+0000","last_deep_scrub":"157'524","last_deep_scrub_stamp":"2024-09-21T22:52:55.869886+0000","last_clean_scrub_stamp":"2024-09-21T22:52:55.869886+0000","objects_scrubbed":17,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:18:43.022680+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0038050610000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3825,"num_read_kb":6408,"num_write":434,"num_write_kb":3443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"157'463","reported_seq":8785,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.599344+0000","last_change":"2024-09-21T22:52:57.599471+0000","last_active":"2024-09-21T22:52:57.599344+0000","last_peered":"2024-09-21T22:52:57.599344+0000","last_clean":"2024-09-21T22:52:57.599344+0000","last_became_active":"2024-09-21T22:50:07.030187+0000","last_became_peered":"2024-09-21T22:50:07.030187+0000","last_unstale":"2024-09-21T22:52:57.599344+0000","last_undegraded":"2024-09-21T22:52:57.599344+0000","last_fullsized":"2024-09-21T22:52:57.599344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'463","last_scrub_stamp":"2024-09-21T22:52:57.599144+0000","last_deep_scrub":"157'463","last_deep_scrub_stamp":"2024-09-21T22:52:57.599144+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.599144+0000","objects_scrubbed":21,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:29.895201+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.004996833,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7939,"num_read_kb":9321,"num_write":346,"num_write_kb":5138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'511","reported_seq":11550,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:00.896760+0000","last_change":"2024-09-21T22:53:00.896850+0000","last_active":"2024-09-21T22:53:00.896760+0000","last_peered":"2024-09-21T22:53:00.896760+0000","last_clean":"2024-09-21T22:53:00.896760+0000","last_became_active":"2024-09-21T22:51:42.459770+0000","last_became_peered":"2024-09-21T22:51:42.459770+0000","last_unstale":"2024-09-21T22:53:00.896760+0000","last_undegraded":"2024-09-21T22:53:00.896760+0000","last_fullsized":"2024-09-21T22:53:00.896760+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'511","last_scrub_stamp":"2024-09-21T22:53:00.896549+0000","last_deep_scrub":"157'511","last_deep_scrub_stamp":"2024-09-21T22:53:00.896549+0000","last_clean_scrub_stamp":"2024-09-21T22:53:00.896549+0000","objects_scrubbed":17,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:10.849065+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0059469450000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":11258,"num_read_kb":43123,"num_write":471,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2289403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"157'516","reported_seq":9816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.011076+0000","last_change":"2024-09-21T22:52:57.011178+0000","last_active":"2024-09-21T22:52:57.011076+0000","last_peered":"2024-09-21T22:52:57.011076+0000","last_clean":"2024-09-21T22:52:57.011076+0000","last_became_active":"2024-09-21T22:52:36.069057+0000","last_became_peered":"2024-09-21T22:52:36.069057+0000","last_unstale":"2024-09-21T22:52:57.011076+0000","last_undegraded":"2024-09-21T22:52:57.011076+0000","last_fullsized":"2024-09-21T22:52:57.011076+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'516","last_scrub_stamp":"2024-09-21T22:52:57.010896+0000","last_deep_scrub":"157'516","last_deep_scrub_stamp":"2024-09-21T22:52:57.010896+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.010896+0000","objects_scrubbed":19,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:16.944282+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0030843009999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":9271,"num_read_kb":22057,"num_write":534,"num_write_kb":15786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":50863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.095680+0000","last_change":"2024-09-21T22:52:57.095734+0000","last_active":"2024-09-21T22:52:57.095680+0000","last_peered":"2024-09-21T22:52:57.095680+0000","last_clean":"2024-09-21T22:52:57.095680+0000","last_became_active":"2024-09-21T22:50:34.211833+0000","last_became_peered":"2024-09-21T22:50:34.211833+0000","last_unstale":"2024-09-21T22:52:57.095680+0000","last_undegraded":"2024-09-21T22:52:57.095680+0000","last_fullsized":"2024-09-21T22:52:57.095680+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:57.095488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:57.095488+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.095488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:29:22.766524+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"157'610","reported_seq":6789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:53.906657+0000","last_change":"2024-09-21T22:52:53.906741+0000","last_active":"2024-09-21T22:52:53.906657+0000","last_peered":"2024-09-21T22:52:53.906657+0000","last_clean":"2024-09-21T22:52:53.906657+0000","last_became_active":"2024-09-21T22:50:07.033777+0000","last_became_peered":"2024-09-21T22:50:07.033777+0000","last_unstale":"2024-09-21T22:52:53.906657+0000","last_undegraded":"2024-09-21T22:52:53.906657+0000","last_fullsized":"2024-09-21T22:52:53.906657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'610","last_scrub_stamp":"2024-09-21T22:52:53.906485+0000","last_deep_scrub":"157'610","last_deep_scrub_stamp":"2024-09-21T22:52:53.906485+0000","last_clean_scrub_stamp":"2024-09-21T22:52:53.906485+0000","objects_scrubbed":12,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:22:42.366158+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0030189599999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5987,"num_read_kb":22288,"num_write":520,"num_write_kb":12134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:55.354703+0000","last_change":"2024-09-21T22:52:55.354752+0000","last_active":"2024-09-21T22:52:55.354703+0000","last_peered":"2024-09-21T22:52:55.354703+0000","last_clean":"2024-09-21T22:52:55.354703+0000","last_became_active":"2024-09-21T22:50:34.194313+0000","last_became_peered":"2024-09-21T22:50:34.194313+0000","last_unstale":"2024-09-21T22:52:55.354703+0000","last_undegraded":"2024-09-21T22:52:55.354703+0000","last_fullsized":"2024-09-21T22:52:55.354703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:55.354582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:55.354582+0000","last_clean_scrub_stamp":"2024-09-21T22:52:55.354582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:04.025195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086298400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'550","reported_seq":4573,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.803717+0000","last_change":"2024-09-21T22:52:57.803873+0000","last_active":"2024-09-21T22:52:57.803717+0000","last_peered":"2024-09-21T22:52:57.803717+0000","last_clean":"2024-09-21T22:52:57.803717+0000","last_became_active":"2024-09-21T22:51:42.456421+0000","last_became_peered":"2024-09-21T22:51:42.456421+0000","last_unstale":"2024-09-21T22:52:57.803717+0000","last_undegraded":"2024-09-21T22:52:57.803717+0000","last_fullsized":"2024-09-21T22:52:57.803717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'550","last_scrub_stamp":"2024-09-21T22:52:57.803482+0000","last_deep_scrub":"157'550","last_deep_scrub_stamp":"2024-09-21T22:52:57.803482+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.803482+0000","objects_scrubbed":19,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:03.513301+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0050121920000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5034,"num_read_kb":6457,"num_write":546,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":532871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.830337+0000","last_change":"2024-09-21T22:52:57.830391+0000","last_active":"2024-09-21T22:52:57.830337+0000","last_peered":"2024-09-21T22:52:57.830337+0000","last_clean":"2024-09-21T22:52:57.830337+0000","last_became_active":"2024-09-21T22:49:56.977710+0000","last_became_peered":"2024-09-21T22:49:56.977710+0000","last_unstale":"2024-09-21T22:52:57.830337+0000","last_undegraded":"2024-09-21T22:52:57.830337+0000","last_fullsized":"2024-09-21T22:52:57.830337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:57.830136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:57.830136+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.830136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:15.949285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010109520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"157'644","reported_seq":3480,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.455387+0000","last_change":"2024-09-21T22:52:57.455498+0000","last_active":"2024-09-21T22:52:57.455387+0000","last_peered":"2024-09-21T22:52:57.455387+0000","last_clean":"2024-09-21T22:52:57.455387+0000","last_became_active":"2024-09-21T22:52:35.325552+0000","last_became_peered":"2024-09-21T22:52:35.325552+0000","last_unstale":"2024-09-21T22:52:57.455387+0000","last_undegraded":"2024-09-21T22:52:57.455387+0000","last_fullsized":"2024-09-21T22:52:57.455387+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'644","last_scrub_stamp":"2024-09-21T22:52:57.455192+0000","last_deep_scrub":"157'644","last_deep_scrub_stamp":"2024-09-21T22:52:57.455192+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.455192+0000","objects_scrubbed":16,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:26.237870+0000","scrub_duration":31,"objects_trimmed":2,"snaptrim_duration":0.0074758790000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2885,"num_read_kb":6451,"num_write":697,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":62364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:54.380570+0000","last_change":"2024-09-21T22:52:54.380607+0000","last_active":"2024-09-21T22:52:54.380570+0000","last_peered":"2024-09-21T22:52:54.380570+0000","last_clean":"2024-09-21T22:52:54.380570+0000","last_became_active":"2024-09-21T22:49:56.971889+0000","last_became_peered":"2024-09-21T22:49:56.971889+0000","last_unstale":"2024-09-21T22:52:54.380570+0000","last_undegraded":"2024-09-21T22:52:54.380570+0000","last_fullsized":"2024-09-21T22:52:54.380570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:54.380452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:54.380452+0000","last_clean_scrub_stamp":"2024-09-21T22:52:54.380452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:16.818349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080498699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"157'595","reported_seq":2835,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:59.576188+0000","last_change":"2024-09-21T22:52:59.576270+0000","last_active":"2024-09-21T22:52:59.576188+0000","last_peered":"2024-09-21T22:52:59.576188+0000","last_clean":"2024-09-21T22:52:59.576188+0000","last_became_active":"2024-09-21T22:51:45.476034+0000","last_became_peered":"2024-09-21T22:51:45.476034+0000","last_unstale":"2024-09-21T22:52:59.576188+0000","last_undegraded":"2024-09-21T22:52:59.576188+0000","last_fullsized":"2024-09-21T22:52:59.576188+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'595","last_scrub_stamp":"2024-09-21T22:52:59.576003+0000","last_deep_scrub":"157'595","last_deep_scrub_stamp":"2024-09-21T22:52:59.576003+0000","last_clean_scrub_stamp":"2024-09-21T22:52:59.576003+0000","objects_scrubbed":16,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:02.132431+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0039760200000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3534,"num_read_kb":13905,"num_write":547,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":120647,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"157'747","reported_seq":7132,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:57.994770+0000","last_change":"2024-09-21T22:52:57.994860+0000","last_active":"2024-09-21T22:52:57.994770+0000","last_peered":"2024-09-21T22:52:57.994770+0000","last_clean":"2024-09-21T22:52:57.994770+0000","last_became_active":"2024-09-21T22:52:38.066909+0000","last_became_peered":"2024-09-21T22:52:38.066909+0000","last_unstale":"2024-09-21T22:52:57.994770+0000","last_undegraded":"2024-09-21T22:52:57.994770+0000","last_fullsized":"2024-09-21T22:52:57.994770+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'747","last_scrub_stamp":"2024-09-21T22:52:57.994578+0000","last_deep_scrub":"157'747","last_deep_scrub_stamp":"2024-09-21T22:52:57.994578+0000","last_clean_scrub_stamp":"2024-09-21T22:52:57.994578+0000","objects_scrubbed":15,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:55.524991+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0041553429999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8231,"num_read_kb":13660,"num_write":662,"num_write_kb":6494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":156987,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"157'3","reported_seq":299,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:56.554755+0000","last_change":"2024-09-21T22:52:56.554821+0000","last_active":"2024-09-21T22:52:56.554755+0000","last_peered":"2024-09-21T22:52:56.554755+0000","last_clean":"2024-09-21T22:52:56.554755+0000","last_became_active":"2024-09-21T22:49:56.976617+0000","last_became_peered":"2024-09-21T22:49:56.976617+0000","last_unstale":"2024-09-21T22:52:56.554755+0000","last_undegraded":"2024-09-21T22:52:56.554755+0000","last_fullsized":"2024-09-21T22:52:56.554755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'3","last_scrub_stamp":"2024-09-21T22:52:56.554561+0000","last_deep_scrub":"157'3","last_deep_scrub_stamp":"2024-09-21T22:52:56.554561+0000","last_clean_scrub_stamp":"2024-09-21T22:52:56.554561+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:52:52.027550+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001833849,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"157'709","reported_seq":5596,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.470271+0000","last_change":"2024-09-21T22:52:58.470364+0000","last_active":"2024-09-21T22:52:58.470271+0000","last_peered":"2024-09-21T22:52:58.470271+0000","last_clean":"2024-09-21T22:52:58.470271+0000","last_became_active":"2024-09-21T22:52:35.325434+0000","last_became_peered":"2024-09-21T22:52:35.325434+0000","last_unstale":"2024-09-21T22:52:58.470271+0000","last_undegraded":"2024-09-21T22:52:58.470271+0000","last_fullsized":"2024-09-21T22:52:58.470271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'709","last_scrub_stamp":"2024-09-21T22:52:58.470057+0000","last_deep_scrub":"157'709","last_deep_scrub_stamp":"2024-09-21T22:52:58.470057+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.470057+0000","objects_scrubbed":14,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:04.748985+0000","scrub_duration":27,"objects_trimmed":6,"snaptrim_duration":0.02221364,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6047,"num_read_kb":19497,"num_write":933,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":25391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:55.409809+0000","last_change":"2024-09-21T22:52:55.409833+0000","last_active":"2024-09-21T22:52:55.409809+0000","last_peered":"2024-09-21T22:52:55.409809+0000","last_clean":"2024-09-21T22:52:55.409809+0000","last_became_active":"2024-09-21T22:52:32.304819+0000","last_became_peered":"2024-09-21T22:52:32.304819+0000","last_unstale":"2024-09-21T22:52:55.409809+0000","last_undegraded":"2024-09-21T22:52:55.409809+0000","last_fullsized":"2024-09-21T22:52:55.409809+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:55.409716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:55.409716+0000","last_clean_scrub_stamp":"2024-09-21T22:52:55.409716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:23.944904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043746300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"157'430","reported_seq":6430,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:56.904212+0000","last_change":"2024-09-21T22:52:56.904295+0000","last_active":"2024-09-21T22:52:56.904212+0000","last_peered":"2024-09-21T22:52:56.904212+0000","last_clean":"2024-09-21T22:52:56.904212+0000","last_became_active":"2024-09-21T22:52:40.086732+0000","last_became_peered":"2024-09-21T22:52:40.086732+0000","last_unstale":"2024-09-21T22:52:56.904212+0000","last_undegraded":"2024-09-21T22:52:56.904212+0000","last_fullsized":"2024-09-21T22:52:56.904212+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'430","last_scrub_stamp":"2024-09-21T22:52:56.904128+0000","last_deep_scrub":"157'430","last_deep_scrub_stamp":"2024-09-21T22:52:56.904128+0000","last_clean_scrub_stamp":"2024-09-21T22:52:56.904128+0000","objects_scrubbed":19,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:57.868749+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001229534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5117,"num_read_kb":47274,"num_write":327,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":23681,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":296,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.791869+0000","last_change":"2024-09-21T22:52:58.791923+0000","last_active":"2024-09-21T22:52:58.791869+0000","last_peered":"2024-09-21T22:52:58.791869+0000","last_clean":"2024-09-21T22:52:58.791869+0000","last_became_active":"2024-09-21T22:49:56.981561+0000","last_became_peered":"2024-09-21T22:49:56.981561+0000","last_unstale":"2024-09-21T22:52:58.791869+0000","last_undegraded":"2024-09-21T22:52:58.791869+0000","last_fullsized":"2024-09-21T22:52:58.791869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T22:52:58.791665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T22:52:58.791665+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.791665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:50:09.846236+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000912899,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"157'741","reported_seq":3276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:00.800573+0000","last_change":"2024-09-21T22:53:00.800681+0000","last_active":"2024-09-21T22:53:00.800573+0000","last_peered":"2024-09-21T22:53:00.800573+0000","last_clean":"2024-09-21T22:53:00.800573+0000","last_became_active":"2024-09-21T22:51:45.473681+0000","last_became_peered":"2024-09-21T22:51:45.473681+0000","last_unstale":"2024-09-21T22:53:00.800573+0000","last_undegraded":"2024-09-21T22:53:00.800573+0000","last_fullsized":"2024-09-21T22:53:00.800573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'741","last_scrub_stamp":"2024-09-21T22:53:00.800333+0000","last_deep_scrub":"157'741","last_deep_scrub_stamp":"2024-09-21T22:53:00.800333+0000","last_clean_scrub_stamp":"2024-09-21T22:53:00.800333+0000","objects_scrubbed":18,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:49.997295+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0037229979999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3993,"num_read_kb":14272,"num_write":854,"num_write_kb":10983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4736618,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"157'2","reported_seq":316,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:00.577138+0000","last_change":"2024-09-21T22:53:00.577214+0000","last_active":"2024-09-21T22:53:00.577138+0000","last_peered":"2024-09-21T22:53:00.577138+0000","last_clean":"2024-09-21T22:53:00.577138+0000","last_became_active":"2024-09-21T22:52:32.316195+0000","last_became_peered":"2024-09-21T22:52:32.316195+0000","last_unstale":"2024-09-21T22:53:00.577138+0000","last_undegraded":"2024-09-21T22:53:00.577138+0000","last_fullsized":"2024-09-21T22:53:00.577138+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'2","last_scrub_stamp":"2024-09-21T22:53:00.576945+0000","last_deep_scrub":"157'2","last_deep_scrub_stamp":"2024-09-21T22:53:00.576945+0000","last_clean_scrub_stamp":"2024-09-21T22:53:00.576945+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:57:30.811562+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00083749400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"157'119","reported_seq":449,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:56.855574+0000","last_change":"2024-09-21T22:52:56.855748+0000","last_active":"2024-09-21T22:52:56.855574+0000","last_peered":"2024-09-21T22:52:56.855574+0000","last_clean":"2024-09-21T22:52:56.855574+0000","last_became_active":"2024-09-21T22:49:53.902457+0000","last_became_peered":"2024-09-21T22:49:53.902457+0000","last_unstale":"2024-09-21T22:52:56.855574+0000","last_undegraded":"2024-09-21T22:52:56.855574+0000","last_fullsized":"2024-09-21T22:52:56.855574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'119","last_scrub_stamp":"2024-09-21T22:52:56.855357+0000","last_deep_scrub":"157'119","last_deep_scrub_stamp":"2024-09-21T22:52:56.855357+0000","last_clean_scrub_stamp":"2024-09-21T22:52:56.855357+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:53.092965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'818","reported_seq":3179,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:59.164542+0000","last_change":"2024-09-21T22:52:59.164706+0000","last_active":"2024-09-21T22:52:59.164542+0000","last_peered":"2024-09-21T22:52:59.164542+0000","last_clean":"2024-09-21T22:52:59.164542+0000","last_became_active":"2024-09-21T22:52:04.079088+0000","last_became_peered":"2024-09-21T22:52:04.079088+0000","last_unstale":"2024-09-21T22:52:59.164542+0000","last_undegraded":"2024-09-21T22:52:59.164542+0000","last_fullsized":"2024-09-21T22:52:59.164542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'818","last_scrub_stamp":"2024-09-21T22:52:59.164327+0000","last_deep_scrub":"157'818","last_deep_scrub_stamp":"2024-09-21T22:52:59.164327+0000","last_clean_scrub_stamp":"2024-09-21T22:52:59.164327+0000","objects_scrubbed":15,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:44.562567+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0014401170000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4434,"num_read_kb":7519,"num_write":1004,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":1632266,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"157'856","reported_seq":6378,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.970522+0000","last_change":"2024-09-21T22:52:58.970615+0000","last_active":"2024-09-21T22:52:58.970522+0000","last_peered":"2024-09-21T22:52:58.970522+0000","last_clean":"2024-09-21T22:52:58.970522+0000","last_became_active":"2024-09-21T22:52:38.063252+0000","last_became_peered":"2024-09-21T22:52:38.063252+0000","last_unstale":"2024-09-21T22:52:58.970522+0000","last_undegraded":"2024-09-21T22:52:58.970522+0000","last_fullsized":"2024-09-21T22:52:58.970522+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'856","last_scrub_stamp":"2024-09-21T22:52:58.970329+0000","last_deep_scrub":"157'856","last_deep_scrub_stamp":"2024-09-21T22:52:58.970329+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.970329+0000","objects_scrubbed":20,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:23.590041+0000","scrub_duration":31,"objects_trimmed":2,"snaptrim_duration":0.0021956359999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7011,"num_read_kb":10391,"num_write":999,"num_write_kb":5649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":63665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'373","reported_seq":2070,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:00.163917+0000","last_change":"2024-09-21T22:53:00.164013+0000","last_active":"2024-09-21T22:53:00.163917+0000","last_peered":"2024-09-21T22:53:00.163917+0000","last_clean":"2024-09-21T22:53:00.163917+0000","last_became_active":"2024-09-21T22:52:39.084006+0000","last_became_peered":"2024-09-21T22:52:39.084006+0000","last_unstale":"2024-09-21T22:53:00.163917+0000","last_undegraded":"2024-09-21T22:53:00.163917+0000","last_fullsized":"2024-09-21T22:53:00.163917+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'373","last_scrub_stamp":"2024-09-21T22:53:00.163728+0000","last_deep_scrub":"157'373","last_deep_scrub_stamp":"2024-09-21T22:53:00.163728+0000","last_clean_scrub_stamp":"2024-09-21T22:53:00.163728+0000","objects_scrubbed":10,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:39:27.650053+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0021614550000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":916,"num_read_kb":14138,"num_write":323,"num_write_kb":3794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":43397,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"157'400","reported_seq":1753,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:55.762740+0000","last_change":"2024-09-21T22:52:55.762857+0000","last_active":"2024-09-21T22:52:55.762740+0000","last_peered":"2024-09-21T22:52:55.762740+0000","last_clean":"2024-09-21T22:52:55.762740+0000","last_became_active":"2024-09-21T22:51:42.452104+0000","last_became_peered":"2024-09-21T22:51:42.452104+0000","last_unstale":"2024-09-21T22:52:55.762740+0000","last_undegraded":"2024-09-21T22:52:55.762740+0000","last_fullsized":"2024-09-21T22:52:55.762740+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'400","last_scrub_stamp":"2024-09-21T22:52:55.762561+0000","last_deep_scrub":"157'400","last_deep_scrub_stamp":"2024-09-21T22:52:55.762561+0000","last_clean_scrub_stamp":"2024-09-21T22:52:55.762561+0000","objects_scrubbed":15,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:43:22.403334+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.003639841,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1421,"num_read_kb":18058,"num_write":333,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":5102978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"157'369","reported_seq":5517,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.558149+0000","last_change":"2024-09-21T22:52:58.558245+0000","last_active":"2024-09-21T22:52:58.558149+0000","last_peered":"2024-09-21T22:52:58.558149+0000","last_clean":"2024-09-21T22:52:58.558149+0000","last_became_active":"2024-09-21T22:51:42.462554+0000","last_became_peered":"2024-09-21T22:51:42.462554+0000","last_unstale":"2024-09-21T22:52:58.558149+0000","last_undegraded":"2024-09-21T22:52:58.558149+0000","last_fullsized":"2024-09-21T22:52:58.558149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'369","last_scrub_stamp":"2024-09-21T22:52:58.557968+0000","last_deep_scrub":"157'369","last_deep_scrub_stamp":"2024-09-21T22:52:58.557968+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.557968+0000","objects_scrubbed":15,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:59:03.766715+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0067205269999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4914,"num_read_kb":18972,"num_write":339,"num_write_kb":8043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2160858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"157'731","reported_seq":4095,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:59.860994+0000","last_change":"2024-09-21T22:52:59.861173+0000","last_active":"2024-09-21T22:52:59.860994+0000","last_peered":"2024-09-21T22:52:59.860994+0000","last_clean":"2024-09-21T22:52:59.860994+0000","last_became_active":"2024-09-21T22:50:07.033115+0000","last_became_peered":"2024-09-21T22:50:07.033115+0000","last_unstale":"2024-09-21T22:52:59.860994+0000","last_undegraded":"2024-09-21T22:52:59.860994+0000","last_fullsized":"2024-09-21T22:52:59.860994+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'731","last_scrub_stamp":"2024-09-21T22:52:59.860743+0000","last_deep_scrub":"157'731","last_deep_scrub_stamp":"2024-09-21T22:52:59.860743+0000","last_clean_scrub_stamp":"2024-09-21T22:52:59.860743+0000","objects_scrubbed":16,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:25:18.427601+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0065980800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6169,"num_read_kb":9182,"num_write":621,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"157'707","reported_seq":7355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:03.981766+0000","last_change":"2024-09-21T22:53:03.981875+0000","last_active":"2024-09-21T22:53:03.981766+0000","last_peered":"2024-09-21T22:53:03.981766+0000","last_clean":"2024-09-21T22:53:03.981766+0000","last_became_active":"2024-09-21T22:52:39.082848+0000","last_became_peered":"2024-09-21T22:52:39.082848+0000","last_unstale":"2024-09-21T22:53:03.981766+0000","last_undegraded":"2024-09-21T22:53:03.981766+0000","last_fullsized":"2024-09-21T22:53:03.981766+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'707","last_scrub_stamp":"2024-09-21T22:53:03.981546+0000","last_deep_scrub":"157'707","last_deep_scrub_stamp":"2024-09-21T22:53:03.981546+0000","last_clean_scrub_stamp":"2024-09-21T22:53:03.981546+0000","objects_scrubbed":18,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:35:19.542548+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0062748129999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7950,"num_read_kb":20755,"num_write":831,"num_write_kb":9364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":203842,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'501","reported_seq":5983,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:02.985871+0000","last_change":"2024-09-21T22:53:02.985985+0000","last_active":"2024-09-21T22:53:02.985871+0000","last_peered":"2024-09-21T22:53:02.985871+0000","last_clean":"2024-09-21T22:53:02.985871+0000","last_became_active":"2024-09-21T22:52:37.065674+0000","last_became_peered":"2024-09-21T22:52:37.065674+0000","last_unstale":"2024-09-21T22:53:02.985871+0000","last_undegraded":"2024-09-21T22:53:02.985871+0000","last_fullsized":"2024-09-21T22:53:02.985871+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'501","last_scrub_stamp":"2024-09-21T22:53:02.985664+0000","last_deep_scrub":"157'501","last_deep_scrub_stamp":"2024-09-21T22:53:02.985664+0000","last_clean_scrub_stamp":"2024-09-21T22:53:02.985664+0000","objects_scrubbed":17,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:53.388307+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0065655490000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6130,"num_read_kb":8986,"num_write":500,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":180986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'659","reported_seq":7592,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:58.157807+0000","last_change":"2024-09-21T22:52:58.157894+0000","last_active":"2024-09-21T22:52:58.157807+0000","last_peered":"2024-09-21T22:52:58.157807+0000","last_clean":"2024-09-21T22:52:58.157807+0000","last_became_active":"2024-09-21T22:50:37.031180+0000","last_became_peered":"2024-09-21T22:50:37.031180+0000","last_unstale":"2024-09-21T22:52:58.157807+0000","last_undegraded":"2024-09-21T22:52:58.157807+0000","last_fullsized":"2024-09-21T22:52:58.157807+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'659","last_scrub_stamp":"2024-09-21T22:52:58.157577+0000","last_deep_scrub":"157'659","last_deep_scrub_stamp":"2024-09-21T22:52:58.157577+0000","last_clean_scrub_stamp":"2024-09-21T22:52:58.157577+0000","objects_scrubbed":16,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:24.118386+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.002843178,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8616,"num_read_kb":38869,"num_write":607,"num_write_kb":9763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":292688,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'982","reported_seq":6628,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:04.967616+0000","last_change":"2024-09-21T22:53:04.967729+0000","last_active":"2024-09-21T22:53:04.967616+0000","last_peered":"2024-09-21T22:53:04.967616+0000","last_clean":"2024-09-21T22:53:04.967616+0000","last_became_active":"2024-09-21T22:52:41.064919+0000","last_became_peered":"2024-09-21T22:52:41.064919+0000","last_unstale":"2024-09-21T22:53:04.967616+0000","last_undegraded":"2024-09-21T22:53:04.967616+0000","last_fullsized":"2024-09-21T22:53:04.967616+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'982","last_scrub_stamp":"2024-09-21T22:53:04.967417+0000","last_deep_scrub":"157'982","last_deep_scrub_stamp":"2024-09-21T22:53:04.967417+0000","last_clean_scrub_stamp":"2024-09-21T22:53:04.967417+0000","objects_scrubbed":14,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:55:00.191814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0030699910000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7340,"num_read_kb":52315,"num_write":1268,"num_write_kb":30202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":94060,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"157'620","reported_seq":8680,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:52:54.922622+0000","last_change":"2024-09-21T22:52:54.922746+0000","last_active":"2024-09-21T22:52:54.922622+0000","last_peered":"2024-09-21T22:52:54.922622+0000","last_clean":"2024-09-21T22:52:54.922622+0000","last_became_active":"2024-09-21T22:50:07.033141+0000","last_became_peered":"2024-09-21T22:50:07.033141+0000","last_unstale":"2024-09-21T22:52:54.922622+0000","last_undegraded":"2024-09-21T22:52:54.922622+0000","last_fullsized":"2024-09-21T22:52:54.922622+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'620","last_scrub_stamp":"2024-09-21T22:52:54.922432+0000","last_deep_scrub":"157'620","last_deep_scrub_stamp":"2024-09-21T22:52:54.922432+0000","last_clean_scrub_stamp":"2024-09-21T22:52:54.922432+0000","objects_scrubbed":20,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:38:17.434461+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0032440780000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9998,"num_read_kb":23254,"num_write":520,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"157'634","reported_seq":8942,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T22:53:01.952752+0000","last_change":"2024-09-21T22:53:01.952906+0000","last_active":"2024-09-21T22:53:01.952752+0000","last_peered":"2024-09-21T22:53:01.952752+0000","last_clean":"2024-09-21T22:53:01.952752+0000","last_became_active":"2024-09-21T22:52:35.323494+0000","last_became_peered":"2024-09-21T22:52:35.323494+0000","last_unstale":"2024-09-21T22:53:01.952752+0000","last_undegraded":"2024-09-21T22:53:01.952752+0000","last_fullsized":"2024-09-21T22:53:01.952752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'634","last_scrub_stamp":"2024-09-21T22:53:01.952566+0000","last_deep_scrub":"157'634","last_deep_scrub_stamp":"2024-09-21T22:53:01.952566+0000","last_clean_scrub_stamp":"2024-09-21T22:53:01.952566+0000","objects_scrubbed":23,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:45.779406+0000","scrub_duration":39,"objects_trimmed":4,"snaptrim_duration":0.015162445,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7416,"num_read_kb":28300,"num_write":636,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":95646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":553,"num_read":220800,"num_read_kb":716986,"num_write":24311,"num_write_kb":331912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5398,"num_bytes_recovered":47060102,"num_keys_recovered":100,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},"log_size":22082,"ondisk_log_size":22082,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":16,"seq":68719476781,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59392,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312448,"statfs":{"total":96636764160,"available":96575946752,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1489208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42428,"kb_used_data":1836,"kb_used_omap":4,"kb_used_meta":40571,"kb_avail":94329412,"statfs":{"total":96636764160,"available":96593317888,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4398,"internal_metadata":41545426},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83008,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288832,"statfs":{"total":96636764160,"available":96551763968,"internally_reserved":0,"allocated":1884160,"data_stored":898859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":1888256,"data_stored":898878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47488,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324352,"statfs":{"total":96636764160,"available":96588136448,"internally_reserved":0,"allocated":2473984,"data_stored":1489208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056661,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39484,"kb_used_data":1836,"kb_used_omap":42,"kb_used_meta":37589,"kb_avail":94332356,"statfs":{"total":96636764160,"available":96596332544,"internally_reserved":0,"allocated":1880064,"data_stored":898840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":38492149},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13946,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T22:53:17.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T22:53:17.783 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T22:53:17.783 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T22:53:17.783 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T22:53:17.783 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:17.872 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T22:53:17.873 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T22:53:17.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:17.957 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T22:53:17.957 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T22:53:17.957 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:18.043 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T22:53:18.044 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T22:53:18.044 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:18.132 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T22:53:18.133 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T22:53:18.133 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:18.224 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T22:53:18.224 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T22:53:18.224 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:22.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-21T22:53:24.326 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T22:53:24.326 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T22:53:24.326 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.377 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T22:53:24.377 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T22:53:24.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.471 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T22:53:24.471 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T22:53:24.471 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T22:53:24.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.500 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T22:53:24.500 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T22:53:24.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.530 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T22:53:24.530 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T22:53:24.531 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T22:53:24.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.563 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T22:53:24.564 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T22:53:24.564 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.629 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T22:53:24.630 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T22:53:24.630 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T22:53:24.653 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T22:53:24.654 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T22:53:24.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-21T22:53:24.690 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:53:24.690 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T22:53:25.006 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:53:25.006 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T22:53:25.217 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:53:25.217 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T22:53:25.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:53:25.406 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T22:53:25.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:53:25.584 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T22:53:25.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:53:25.843 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T22:53:26.079 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:53:26.079 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T22:53:26.251 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:53:26.251 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T22:53:26.411 INFO:tasks.ceph:Archiving mon data... 2024-09-21T22:53:26.413 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/data/mon.a.tgz 2024-09-21T22:53:26.413 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-21T22:53:26.419 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.HT3uTMmXdW 2024-09-21T22:53:26.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.HT3uTMmXdW 2024-09-21T22:53:26.563 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.HT3uTMmXdW 2024-09-21T22:53:26.627 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.HT3uTMmXdW is 401KB 2024-09-21T22:53:26.685 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.HT3uTMmXdW 2024-09-21T22:53:26.690 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/data/mon.c.tgz 2024-09-21T22:53:26.690 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-21T22:53:26.735 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.OvTdhCFhLH 2024-09-21T22:53:26.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OvTdhCFhLH 2024-09-21T22:53:26.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.OvTdhCFhLH 2024-09-21T22:53:26.970 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.OvTdhCFhLH is 540KB 2024-09-21T22:53:26.988 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.OvTdhCFhLH 2024-09-21T22:53:26.992 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/data/mon.b.tgz 2024-09-21T22:53:26.992 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-21T22:53:26.998 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.k7hutOWdTU 2024-09-21T22:53:26.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.k7hutOWdTU 2024-09-21T22:53:27.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.k7hutOWdTU 2024-09-21T22:53:27.228 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.k7hutOWdTU is 540KB 2024-09-21T22:53:27.246 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.k7hutOWdTU 2024-09-21T22:53:27.250 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T22:53:27.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T22:53:27.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T22:53:27.311 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T22:53:27.348 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T22:53:27.349 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T22:53:27.350 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi094/crash 2024-09-21T22:53:27.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T22:53:27.366 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi099/crash 2024-09-21T22:53:27.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T22:53:27.405 INFO:tasks.ceph:Compressing logs... 2024-09-21T22:53:27.405 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T22:53:27.416 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T22:53:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35035.log 2024-09-21T22:53:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31489.log 2024-09-21T22:53:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35035.log.gz 2024-09-21T22:53:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42990.log 2024-09-21T22:53:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33507.log 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31489.log.gz 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42990.log.gz 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33507.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33507.log.gz 2024-09-21T22:53:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31550.log 2024-09-21T22:53:27.445 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31550.log.gz 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-09-21T22:53:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31754.log 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31754.log.gz 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43314.log 2024-09-21T22:53:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33441.log 2024-09-21T22:53:27.449 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-09-21T22:53:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26380.log 2024-09-21T22:53:27.449 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43314.log.gz 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33441.log.gz 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.26380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33511.log 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26380.log.gz 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T22:53:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33511.log.gz 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32393.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32393.log.gz 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35196.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-09-21T22:53:27.453 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35196.log.gz 2024-09-21T22:53:27.457 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42595.log 2024-09-21T22:53:27.457 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2024-09-21T22:53:27.457 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-09-21T22:53:27.457 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42595.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43121.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34475.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43121.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34475.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.37181.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34475.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31968.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/objectstore_tool.37181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32167.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31968.log.gz 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43318.log 2024-09-21T22:53:27.458 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32167.log.gz 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33758.log 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43900.log 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43318.log.gz 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33758.log.gz 2024-09-21T22:53:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-09-21T22:53:27.462 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41468.log 2024-09-21T22:53:27.462 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43900.log.gz 2024-09-21T22:53:27.462 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-09-21T22:53:27.462 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33333.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41468.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33333.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43718.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.37181.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.43718.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42626.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43718.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-09-21T22:53:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42626.log.gz 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42834.log 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.35436.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43874.log 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40447.log 2024-09-21T22:53:27.464 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42834.log.gz 2024-09-21T22:53:27.467 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32733.log 2024-09-21T22:53:27.467 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43874.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40447.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43119.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32733.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.43119.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42099.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43119.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31551.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34319.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42099.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31551.log.gz 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43666.log 2024-09-21T22:53:27.468 INFO:teuthology.orchestra.run.smithi094.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.34319.log.gz 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43068.log 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5/var/log/ceph/ceph-client.admin.43666.log: --verbose -- /var/log/ceph/ceph-client.admin.40477.log 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43666.log.gz 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43068.log.gz 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33635.log 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40477.log.gz 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2024-09-21T22:53:27.469 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33635.log.gz 2024-09-21T22:53:27.475 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34520.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26507.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/objectstore_tool.34520.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26507.log.gz 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26411.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26531.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26411.log.gz 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26435.log 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26531.log.gz 2024-09-21T22:53:27.476 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32211.log 2024-09-21T22:53:27.477 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26435.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T22:53:27.477 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26435.log.gz 2024-09-21T22:53:27.477 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26387.log 2024-09-21T22:53:27.477 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/objectstore_tool.32211.log: /var/log/ceph/ceph.audit.log: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T22:53:27.477 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.34520.log.gz 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26562.log 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26387.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33761.log 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26387.log.gz 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26562.log.gz 2024-09-21T22:53:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26459.log 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/objectstore_tool.33761.log: /var/log/ceph/ceph-client.admin.26459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26483.log 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26459.log.gz 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.b.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.32211.log.gz 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.26483.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32972.log 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26483.log.gz 2024-09-21T22:53:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/objectstore_tool.32972.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.33761.log.gz 2024-09-21T22:53:27.489 INFO:teuthology.orchestra.run.smithi099.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.32972.log.gz 2024-09-21T22:53:27.496 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-09-21T22:53:27.496 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31777.log 2024-09-21T22:53:27.496 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31777.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35099.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26636.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35099.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34504.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.26636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26636.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34504.log.gz 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44108.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31983.log 2024-09-21T22:53:27.497 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44108.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44108.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31983.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42782.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31559.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42782.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31559.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31969.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-09-21T22:53:27.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34152.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31969.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34152.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32395.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34213.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34176.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32395.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31463.log 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34213.log.gz 2024-09-21T22:53:27.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34176.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43770.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31463.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35385.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43770.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42004.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40481.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35385.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42004.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31568.log 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40481.log.gz 2024-09-21T22:53:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43313.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31568.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35129.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40384.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43313.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35129.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40384.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33598.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.36356.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33598.log.gz 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31765.log 2024-09-21T22:53:27.501 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/objectstore_tool.36356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31443.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31765.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31765.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31443.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31443.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32945.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32945.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32945.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43118.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32219.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31558.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43118.log.gz 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41528.log 2024-09-21T22:53:27.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32219.log: /var/log/ceph/ceph-client.admin.31558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32219.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31558.log.gz/var/log/ceph/ceph-client.admin.41528.log: 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32637.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41528.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39062.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34641.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32637.log: /var/log/ceph/ceph-client.admin.39062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32637.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39062.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34641.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.36356.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34100.log 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34641.log.gz 2024-09-21T22:53:27.503 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34100.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34906.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40141.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44056.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35359.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40141.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33849.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44056.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41154.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35359.log: /var/log/ceph/ceph-client.admin.33849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34863.log 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35359.log.gz 2024-09-21T22:53:27.504 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33849.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41154.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40356.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43692.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34863.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32708.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32852.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43692.log.gz/var/log/ceph/ceph-client.admin.32708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41724.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32708.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32852.log.gz 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40956.log 2024-09-21T22:53:27.505 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41784.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41724.log: /var/log/ceph/ceph-client.admin.40956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44082.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41724.log.gz 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40956.log.gz 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41784.log.gz 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42509.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44082.log.gz 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34839.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42509.log: /var/log/ceph/ceph-client.admin.34839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-09-21T22:53:27.513 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34839.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42509.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33303.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39676.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43312.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33303.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39676.log: /var/log/ceph/ceph-client.admin.43312.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39676.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43312.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36134.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33176.log 2024-09-21T22:53:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42318.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36134.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33176.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40653.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32385.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42318.log: /var/log/ceph/ceph-client.admin.40653.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42318.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40653.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.32385.log: 95.5% -- replaced with /var/log/ceph/ceph-client.0.32385.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42131.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21529.log 2024-09-21T22:53:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41494.log: /var/log/ceph/ceph-client.admin.42131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42131.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.tmp-client.admin.21529.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37593.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.21529.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34764.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32271.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37593.log: /var/log/ceph/ceph-client.admin.34764.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37593.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34764.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32271.log.gz 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33252.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42001.log 2024-09-21T22:53:27.516 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40608.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33252.log.gz 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31967.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42001.log: /var/log/ceph/ceph-client.admin.40608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42467.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42001.log.gz 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40608.log.gz 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31967.log.gz 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42467.log: 0.0%/var/log/ceph/ceph-client.admin.43848.log: -- replaced with /var/log/ceph/ceph-client.admin.42467.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:27.522 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41376.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44158.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41376.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44158.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34944.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42652.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34944.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33701.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34944.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42652.log.gz 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41421.log 2024-09-21T22:53:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42886.log 2024-09-21T22:53:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33701.log.gz 2024-09-21T22:53:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41421.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-09-21T22:53:27.524 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41421.log.gz 2024-09-21T22:53:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42886.log.gz 2024-09-21T22:53:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40527.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40238.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40527.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31406.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40527.log.gz 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40238.log.gz 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31562.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31406.log: /var/log/ceph/ceph.audit.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31406.log.gz 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43016.log 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31562.log.gz 2024-09-21T22:53:27.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33784.log 2024-09-21T22:53:27.527 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43016.log.gz 2024-09-21T22:53:27.527 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43536.log 2024-09-21T22:53:27.527 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33784.log.gz 2024-09-21T22:53:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-09-21T22:53:27.528 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T22:53:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43536.log.gz 2024-09-21T22:53:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30692.log 2024-09-21T22:53:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32516.log 2024-09-21T22:53:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40707.log: /var/log/ceph/ceph-client.admin.30692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-09-21T22:53:27.529 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30692.log.gz 2024-09-21T22:53:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38003.log 2024-09-21T22:53:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2024-09-21T22:53:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32516.log: /var/log/ceph/objectstore_tool.38003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32516.log.gz 2024-09-21T22:53:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2024-09-21T22:53:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2024-09-21T22:53:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31982.log 2024-09-21T22:53:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2024-09-21T22:53:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33151.log 2024-09-21T22:53:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31982.log.gz 2024-09-21T22:53:27.534 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40789.log 2024-09-21T22:53:27.534 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33151.log.gz 2024-09-21T22:53:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-09-21T22:53:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40789.log.gz 2024-09-21T22:53:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-21T22:53:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 2024-09-21T22:53:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34273.log 2024-09-21T22:53:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42557.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.38003.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-21T22:53:27.538 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40763.log 2024-09-21T22:53:27.539 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43315.log 2024-09-21T22:53:27.539 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34273.log: /var/log/ceph/ceph-client.admin.40763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34273.log.gz 2024-09-21T22:53:27.539 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40763.log.gz 2024-09-21T22:53:27.539 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31775.log 2024-09-21T22:53:27.540 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34126.log 2024-09-21T22:53:27.540 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43315.log: /var/log/ceph/ceph-client.admin.31775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43315.log.gz 2024-09-21T22:53:27.540 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31775.log.gz 2024-09-21T22:53:27.540 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-21T22:53:27.541 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32999.log 2024-09-21T22:53:27.541 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34126.log.gz 2024-09-21T22:53:27.542 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34724.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-09-21T22:53:27.542 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-09-21T22:53:27.542 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32999.log.gz 2024-09-21T22:53:27.542 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40793.log 2024-09-21T22:53:27.542 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43744.log 2024-09-21T22:53:27.543 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-09-21T22:53:27.544 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40793.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34584.log 2024-09-21T22:53:27.544 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40793.log.gz 2024-09-21T22:53:27.544 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43744.log.gz 2024-09-21T22:53:27.544 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26528.log 2024-09-21T22:53:27.544 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-09-21T22:53:27.545 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34584.log.gz 2024-09-21T22:53:27.545 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.26528.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34400.log 2024-09-21T22:53:27.545 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26528.log.gz 2024-09-21T22:53:27.546 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-09-21T22:53:27.546 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41027.log 2024-09-21T22:53:27.547 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43952.log 2024-09-21T22:53:27.547 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34400.log: /var/log/ceph/ceph-client.admin.41027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34400.log.gz 2024-09-21T22:53:27.547 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41027.log.gz 2024-09-21T22:53:27.547 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31569.log 2024-09-21T22:53:27.548 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43952.log.gz 2024-09-21T22:53:27.548 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40894.log 2024-09-21T22:53:27.549 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41974.log 2024-09-21T22:53:27.549 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31569.log: /var/log/ceph/ceph-client.admin.40894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31569.log.gz 2024-09-21T22:53:27.549 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40894.log.gz 2024-09-21T22:53:27.549 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31957.log 2024-09-21T22:53:27.550 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42808.log 2024-09-21T22:53:27.550 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41974.log: /var/log/ceph/ceph-client.admin.31957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41974.log.gz 2024-09-21T22:53:27.550 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31957.log.gz 2024-09-21T22:53:27.550 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41818.log 2024-09-21T22:53:27.551 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39465.log 2024-09-21T22:53:27.551 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42808.log: /var/log/ceph/ceph-client.admin.41818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42808.log.gz 2024-09-21T22:53:27.551 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41818.log.gz 2024-09-21T22:53:27.551 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43926.log 2024-09-21T22:53:27.552 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31567.log 2024-09-21T22:53:27.552 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39465.log: /var/log/ceph/ceph-client.admin.43926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39465.log.gz 2024-09-21T22:53:27.552 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43926.log.gz 2024-09-21T22:53:27.552 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-09-21T22:53:27.553 INFO:teuthology.orchestra.run.smithi099.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T22:53:27.553 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31567.log.gz 2024-09-21T22:53:27.553 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-21T22:53:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2024-09-21T22:53:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-09-21T22:53:27.555 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42505.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43042.log 2024-09-21T22:53:27.555 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-21T22:53:27.555 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2024-09-21T22:53:27.555 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38029.log 2024-09-21T22:53:27.555 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31764.log 2024-09-21T22:53:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43042.log.gz 2024-09-21T22:53:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-09-21T22:53:27.556 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38029.log.gz 2024-09-21T22:53:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31764.log.gz 2024-09-21T22:53:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32886.log 2024-09-21T22:53:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31776.log 2024-09-21T22:53:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-09-21T22:53:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32886.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.32886.log.gz -5 --verbose -- /var/log/ceph/objectstore_tool.35503.log 2024-09-21T22:53:27.558 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31776.log.gz 2024-09-21T22:53:27.559 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32193.log 2024-09-21T22:53:27.559 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43822.log 2024-09-21T22:53:27.560 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/objectstore_tool.35503.log: /var/log/ceph/ceph-client.admin.32193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32193.log.gz 2024-09-21T22:53:27.566 INFO:teuthology.orchestra.run.smithi094.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.35503.log.gz 2024-09-21T22:53:27.566 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38969.log 2024-09-21T22:53:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-09-21T22:53:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43822.log.gz 2024-09-21T22:53:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38969.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-09-21T22:53:27.568 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38969.log.gz 2024-09-21T22:53:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-09-21T22:53:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-09-21T22:53:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40856.log 2024-09-21T22:53:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-09-21T22:53:27.570 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42730.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43614.log 2024-09-21T22:53:27.570 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-09-21T22:53:27.570 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40856.log.gz 2024-09-21T22:53:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35440.log 2024-09-21T22:53:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34500.log 2024-09-21T22:53:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43614.log.gz 2024-09-21T22:53:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-09-21T22:53:27.571 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35440.log.gz 2024-09-21T22:53:27.572 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34500.log.gz 2024-09-21T22:53:27.572 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31379.log 2024-09-21T22:53:27.573 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33947.log 2024-09-21T22:53:27.573 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43562.log: /var/log/ceph/ceph-client.admin.31379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-09-21T22:53:27.573 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31379.log.gz 2024-09-21T22:53:27.573 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31750.log 2024-09-21T22:53:27.574 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-09-21T22:53:27.574 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33947.log: /var/log/ceph/ceph-client.admin.31750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33947.log.gz 2024-09-21T22:53:27.574 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31750.log.gz 2024-09-21T22:53:27.574 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38872.log 2024-09-21T22:53:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42182.log 2024-09-21T22:53:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-09-21T22:53:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38872.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2024-09-21T22:53:27.576 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38872.log.gz 2024-09-21T22:53:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42182.log.gz 2024-09-21T22:53:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40570.log 2024-09-21T22:53:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32353.log 2024-09-21T22:53:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2024-09-21T22:53:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40570.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.40570.log.gz --verbose 2024-09-21T22:53:27.577 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.31984.log 2024-09-21T22:53:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32353.log.gz 2024-09-21T22:53:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44030.log 2024-09-21T22:53:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31985.log 2024-09-21T22:53:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31984.log: /var/log/ceph/ceph-client.admin.44030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31984.log.gz 2024-09-21T22:53:27.579 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44030.log.gz 2024-09-21T22:53:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31758.log 2024-09-21T22:53:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42964.log 2024-09-21T22:53:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.31985.log: /var/log/ceph/ceph-client.admin.31758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31985.log.gz 2024-09-21T22:53:27.580 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31758.log.gz 2024-09-21T22:53:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-09-21T22:53:27.581 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32805.log 2024-09-21T22:53:27.581 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42964.log.gz 2024-09-21T22:53:27.582 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2024-09-21T22:53:27.582 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-09-21T22:53:27.582 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32805.log.gz 2024-09-21T22:53:27.582 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35272.log 2024-09-21T22:53:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33948.log 2024-09-21T22:53:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34363.log: /var/log/ceph/ceph-client.admin.35272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2024-09-21T22:53:27.583 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35272.log.gz 2024-09-21T22:53:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33042.log 2024-09-21T22:53:27.584 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32595.log 2024-09-21T22:53:27.584 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33948.log.gz 2024-09-21T22:53:27.584 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33042.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35300.log 2024-09-21T22:53:27.585 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33042.log.gz 2024-09-21T22:53:27.585 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.32595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32595.log.gz 2024-09-21T22:53:27.585 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35300.log.gz 2024-09-21T22:53:27.608 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T22:53:27.670 INFO:teuthology.orchestra.run.smithi094.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T22:53:27.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-21T22:53:27.775 INFO:teuthology.orchestra.run.smithi094.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T22:53:27.849 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T22:53:28.502 INFO:teuthology.orchestra.run.smithi099.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T22:53:29.937 INFO:teuthology.orchestra.run.smithi094.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T22:53:30.548 INFO:teuthology.orchestra.run.smithi099.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T22:53:30.982 INFO:teuthology.orchestra.run.smithi094.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T22:53:30.997 INFO:teuthology.orchestra.run.smithi099.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T22:53:31.245 INFO:teuthology.orchestra.run.smithi094.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T22:53:31.246 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-21T22:53:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m3.828s 2024-09-21T22:53:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m10.400s 2024-09-21T22:53:31.246 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.486s 2024-09-21T22:53:31.837 INFO:teuthology.orchestra.run.smithi099.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T22:53:31.839 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-21T22:53:31.840 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m4.387s 2024-09-21T22:53:31.840 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m11.787s 2024-09-21T22:53:31.840 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.486s 2024-09-21T22:53:31.841 INFO:tasks.ceph:Archiving logs... 2024-09-21T22:53:31.841 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi094/log 2024-09-21T22:53:31.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T22:53:32.952 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi099/log 2024-09-21T22:53:32.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T22:53:34.070 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T22:53:34.079 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T22:53:34.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T22:53:34.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T22:53:34.164 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T22:53:34.172 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T22:53:34.172 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T22:53:34.175 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T22:53:34.185 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T22:53:34.185 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-21T22:53:34.185 INFO:teuthology.orchestra.run.smithi094.stdout:*hv01.front.sepi 67.205.162.81 3 u 51 64 77 0.082 +0.344 0.913 2024-09-21T22:53:34.186 INFO:teuthology.orchestra.run.smithi094.stdout:+hv02.front.sepi 63.231.80.2 3 u 53 64 77 0.077 +2.158 0.876 2024-09-21T22:53:34.186 INFO:teuthology.orchestra.run.smithi094.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 77 0.096 -1.496 0.700 2024-09-21T22:53:34.186 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout:*hv01.front.sepi 67.205.162.81 3 u 56 64 77 0.103 -0.781 0.635 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout:+hv02.front.sepi 63.231.80.2 3 u 49 64 77 0.122 +1.574 0.875 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout:+hv03.front.sepi 74.6.168.72 3 u 59 64 77 0.209 -2.177 0.660 2024-09-21T22:53:34.187 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T22:53:34.188 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T22:53:34.196 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T22:53:34.197 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T22:53:34.204 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T22:53:34.211 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T22:53:34.219 INFO:teuthology.task.internal:Duration was 838.305443 seconds 2024-09-21T22:53:34.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T22:53:34.227 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T22:53:34.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T22:53:34.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T22:53:34.266 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T22:53:34.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-21T22:53:34.267 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T22:53:34.320 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T22:53:34.320 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T22:53:34.330 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T22:53:34.331 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T22:53:34.364 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T22:53:34.400 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T22:53:34.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T22:53:34.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T22:53:34.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T22:53:34.515 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T22:53:34.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T22:53:34.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T22:53:34.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T22:53:34.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T22:53:34.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T22:53:34.580 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-21T22:53:34.589 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-21T22:53:34.750 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T22:53:34.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:53:34.795 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T22:53:34.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:53:34.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T22:53:34.810 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T22:53:34.810 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi094 2024-09-21T22:53:34.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T22:53:34.864 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914492/remote/smithi099 2024-09-21T22:53:34.864 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T22:53:34.885 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T22:53:34.885 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T22:53:34.904 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T22:53:34.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T22:53:34.940 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T22:53:34.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T22:53:34.967 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T22:53:34.967 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T22:53:34.969 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T22:53:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: 655474 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 22:53 /home/ubuntu/cephtest 2024-09-21T22:53:34.977 INFO:teuthology.orchestra.run.smithi099.stdout: 658615 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 22:53 /home/ubuntu/cephtest 2024-09-21T22:53:34.978 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T22:53:34.989 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T22:53:35.025 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 838.3054430484772 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T22:53:35.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T22:53:35.119 INFO:teuthology.run:pass